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Usability  of  Information  in 
Battle  Management  Operations 

(RTO  MP-57) 


Executive  Summary 

On  10-13  April  2000,  NATO,  Partnership  for  Peace,  and  Non-NATO  nationals  from  21  countries  met 
in  Oslo,  Norway  to  discuss  the  perceptual,  cognitive,  social,  and  contextual  factors  and  considerations 
that  will  impact  the  usefulness  and  usability  of  information  and  information  technologies  in  battle 
management  operations.  This  symposium  was  part  of  the  activities  of  the  Human  Factors  and  Medicine 
Panel  of  the  North  Atlantic  Treaty  Organization’s  Research  and  Technology  Organization. 

Command  and  control  operations  are  becoming  increasingly  more  complex.  At  the  same  time, 
information  technology  is  evolving  at  an  unprecedented  pace  yielding  impressive  capabilities  but  also 
helping  to  place  a  tremendous  informational  and  perceptual  burden  on  commanders,  controllers,  and 
warfighters.  Past  history  suggests  that  when  system  developers  fail  to  consider  the  mutual  effects  of 
cognitive,  informational,  social,  and  contextual  factors,  there  is  an  increased  probability  of  incidents, 
accidents,  and  failures.  By  contrast,  with  new  collaboration  and  visualization  technologies,  it  is 
possible  to  increase  the  perceptual,  cognitive,  and  information  utility  of  command  and  control  systems. 

To  achieve  this  improved  utility  in  battle  management  operations,  it  is  imperative  that  command  and 
control  system  designers  interweave  collaboration  and  visualization  technologies  with  a  deep 
understanding  of  how  humans  perceive  and  process  information,  make  decisions,  and  interact  with 
computer  interfaces,  and  how  users  function  in  individual  and  collaborative  environments.  This 
combination  of  knowledge  of  human  capabilities  with  technology  advances  offers  the  promise  of 
providing  mission-  and  task-critical  information  that  is  easily  used  by  battlespace  managers  and 
warfighters. 
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PExploitation  de  l’information  dans  les  operations  de 

gestion  du  champ  de  bataille 

(RTO  MP-57) 


Synthese 

Des  membres  de  l’OTAN,  des  representants  des  pays  du  Partenariat  pour  la  Paix  et  des  representants  de 
21  pays  non-membres  de  l’OTAN  se  sont  reunis  a  Oslo  du  10  au  13  avril  2000  pour  discuter  des 
facteurs  cognitifs,  perceptifs,  sociaux  et  conjoncturels  qui  auront  un  impact  sur  l’interet  et 
l’exploitabilite  des  informations  et  des  technologies  de  l’information  dans  le  domaine  du 
commandement  et  controle  du  champ  de  bataille.  Ce  symposium  faisait  partie  des  activites  de  la 
Commission  des  Facteurs  humains  et  de  la  Medecine  de  1’ Organisation  pour  la  recherche  et  la 
technologie  de  l’OTAN  (RTO/HFM). 

Les  operations  de  commandement  et  controle  deviennent  de  plus  en  plus  complexes.  Parallelement,  les 
technologies  de  F  information  evoluent  a  une  vitesse  sans  precedent,  offrant  certes  des  capacites 
impressionnantes,  mais  representant  aussi  une  charge  informationnelle  et  perceptive  considerable  pour 
les  chefs  militaires,  les  controleurs  et  les  combattants.  L’experience  du  passe  indique  que  pour  les 
concepteurs  de  systemes,  le  fait  de  ne  pas  prendre  en  compte  les  effets  combines  des  differents  facteurs 
cognitifs,  informationnels,  sociaux  et  conjoncturels  equivaut  a  augmenter  la  probabilite  d’incidents, 
d’ accidents  et  de  pannes.  En  revanche,  de  nouvelles  technologies  de  cooperation  et  de  visualisation 
permettent  d’accroitre  l’utilite  perceptive,  cognitive  et  informationnelle  des  systemes  de 
commandement  et  controle. 

Afin  d’ assurer  une  meilleure  efficacite  dans  la  gestion  du  champ  de  bataille,  il  est  indispensable 
d’integrer  dans  les  operations  de  commandement  et  controle  et  dans  les  recherches  technologiques,  une 
comprehension  approfondie  des  capacites  cognitives  et  perceptives  humaines  (c’est  a  dire  la  maniere 
dont  les  etres  humains  pergoivent  et  traitent  F information,  prennent  des  decisions,  et  interagissent  avec 
les  interfaces  informatiques)  ainsi  que  la  connaissance  du  comportement  humain  dans  des 
environnements  individuels  et  cooperatifs.  Cette  combinaison  des  capacites  humaines  et  des  avancees 
technologiques  permet  d’esperer  disposer  d’ informations  decisives  pour  la  mission  sous  une  forme 
facilement  exploitable  par  les  gestionnaires  du  champ  de  bataille  et  les  combattants. 
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INTRODUCTION 

On  10-13  April  2000,  NATO,  Partnership  for  Peace,  and 
Non-NATO  nationals  from  21  countries  met  in  Oslo, 
Norway  to  discuss  the  perceptual,  cognitive,  social,  and 
contextual  factors  and  considerations  that  will  impact  the 
usefulness  and  usability  of  information  and  information 
technologies  in  battle  management  operations.  Sponsored 
by  the  Human  Factors  and  Medicine  Panel  of  the  North 
Atlantic  Treaty  Organization's  Research  and  Technology 
Organization,  the  symposium  participants  discussed  the 
problem,  research  approaches  and  techniques  for 
improving  team  performance  and  enhancing 
effectiveness,  concepts  for  battlespace  visualization  and 
decision  support,  and  the  integration  of  collaborative 
battle  management  systems. 

THEME  /  OVERVIEW 

Command  and  control  operations  are  becoming 
increasingly  more  complex.  At  the  same  time, 
information  technology  is  evolving  at  an  unprecedented 
pace,  yielding  impressive  capabilities  but  also  helping  to 
place  a  tremendous  informational  and  perceptual  burden 
on  commanders,  controllers,  and  warfighters.  Past  history 
suggests  that  when  system  developers  fail  to  consider  the 
mutual  effects  of  cognitive,  informational,  social,  and 
contextual  factors,  there  is  an  increased  probability  of 
incidents,  accidents,  and  failures.  By  contrast,  with  new 
collaboration  and  visualization  technologies,  it  should  be 
possible  to  increase  the  perceptual,  cognitive,  and 
information  utility  of  command  and  control  systems. 

Collaboration  technologies  enable  people  to  share 
information,  communicate,  and  coordinate  across  time 
and  distance  boundaries.  They  include  asynchronous 
tools,  real-time  conferencing  tools,  and  virtual  collocation 
tools  (often  referred  to  as  place-based  collaboration 
environments).  Visualization  technologies  can  help 
humans  perceive  and  process  information.  They  include 
information  visualization,  graphic  databases,  multimedia, 
animation,  video  information  storage  and  retrieval,  etc. 
The  promise  of  both  sets  of  technologies  is  to  improve  our 
ability  to  collaborate,  coordinate,  share,  and  understand 
information  to  facilitate  inter-  and  intra-organizational 
teams. 

To  achieve  this  improved  utility  in  battle  management 
operations,  it  is  imperative  that  command  and  control 
system  designers  interweave  collaboration  and 


visualization  technologies  with  a  deep  understanding  of 
how  humans  perceive  and  process  information,  make 
decisions,  and  interact  with  computer  interfaces,  and 
how  users  function  in  individual  and  collaborative 
environments.  This  combination  of  knowledge  of 
human  capabilities  with  technology  advances  offers  the 
promise  of  providing  mission-  and  task-critical 
information  that  is  easily  used  by  battlespace  managers 
and  warfighters. 

SYMPOSIUM  PROGRAM 

The  Symposium,  chaired  by  Dr.  K.  Boff  and  Dr.  N. 
Gershon  from  the  USA,  consisted  of  an  Opening 
Session,  five  Scientific  Sessions,  four  Keynote 
Addresses,  22  papers,  a  Capstone  Panel,  and  an  open 
discussion  period.  The  final  program  was  organized  by 
Ms.  J.  Davies  (UK),  Dr.  B.  Doring  (GE),  Dr.  E.  Fosse 
(NO),  Dr.  N.  Gershon  (USA),  Dr.  J.  Grau  (FR),  Dr.  A. 
Leger  (FR),  Dr.  G.  Rood  (UK),  Dr.  K.  Savasan  (TU), 
and  Dr.  H.  Schuffel  (NE). 

After  welcoming  addresses  by  Gen.  S.  Frisyold,  Chief 
of  Defense,  Norway,  and  Maj.  Gen.  L.  Rosen,  Director 
General,  Joint  Medical  Services,  Norway,  the  Opening 
Session  concluded  with  a  Keynote  Address  (I)  by  BG 
K.  Alexander  (USA). 

Dr.  G.  Rood  (UK)  chaired  Session  I,  Operational 
Problems  in  Battlespace  Management ,  featuring  a 
Keynote  Address  (II)  by  Lt.  Col.  G.  Pugh  (UK). 

Session  II,  Research  Approaches ,  was  conducted  in 
two  parts:  Ms.  J.  Davies  (UK)  and  Dr.  B.  Doring  (GE) 
chaired  Session  Ha,  Team  Performance ,  and  Dr.  H. 
Schuffel  (NE)  and  Dr.  J.  Grau  (FR)  chaired  Session  lib, 
Techniques  for  Enhancing  Effectiveness . 

Dr.  A.  Leger  (FR)  and  Dr.  H.  Schuffel  (NE)  chaired 
Session  III,  Concepts  for  Battlespace  Visualization  and 
Decision  Support ,  featuring  a  Keynote  Address  (III)  by 
Gen.  L.  Franquart  (FR). 

Dr.  E.  Fosse  (NO)  and  Dr.  N.  Gershon  (USA)  chaired 
Session  IV,  Decision  Support  Considerations ,  featuring 
a  Keynote  Address  (IV)  by  Prof.  R.  Vasmes  (NO). 

Dr.  Boff  and  Ms.  S.  McFadden  (CA)  chaired 
Session  V,  Integration  and  Test  of  Battle  Management 
Systems.  Dr.  Gershon  and  Dr.  Boff  chaired  the 
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Capstone  Panel,  and  Dr.  Gershon  moderated  an  open 
discussion  that  took  place  earlier  in  the  symposium. 

TECHNICAL  EVALUATION 
I:  The  Problem 

Problem  Statement:  Information  rules  the  battlespace, 

Dr.  Boff  began  the  symposium.  In  the  future,  bits  and 
bytes  may  be  more  important  to  determining  the  outcome 
of  conflict  than  bombs  and  bullets.  To  do  so,  information 
must  be  both  useful  (able  to  promote  advantage)  and 
useable  (accessed,  interpreted,  and  applied  for  its 
purpose).  The  problem,  according  to  BG  Alexander 
(Keynote  I),  is  how  to  visualize  information  to  plan  and 
fight — or  to  prevent — wars.  Currently,  we  are  unable  to 
share  and  use  information  (national,  theater,  and  tactical) 
efficiently,  across  echelons  between  services  (Army, 

Navy,  Air  Force,  Marine  Corps),  and  amongst  coalition 
partners.  The  problem  derives  from  a  number  of  causes, 
including:  coalition  limitations;  formatting  differences; 
security  classification  requirements;  differing  timeliness 
requirements  for  information;  lack  of  adequate  processing 
and  collaboration  capabilities;  lack  of  independent 
systems.  As  a  result,  information  hits  different 
organizations  at  different  times  in  different  formats, 
resulting  in  disjointed,  poorly  coordinated  and  differing 
reactions  to  a  common  situation.  This  requires  extensive 
and  often  inefficient  manual  efforts  to  ensure  we  are 
coordinated  at  every  echelon. 

The  potential  for  misunderstanding  is  high  when 
everyone's  picture  differs  slightly.  As  Maj.  Gen.  Rosen 
noted  in  his  welcoming  address,  preparedness  and 
efficiency  and  the  ability  to  work  across  borders  are 
critical  to  success  in  dealing  with  human  conflicts  and 
disasters.  In  this  modem  world  of  technology, 
information,  education,  and  training  are  all  critical  to 
enabling  coalition  forces  to  operate  effectively,  and  as 
efficiently  and  safely  as  possible. 

The  challenge,  Alexander  suggests,  is  to  get  the  right  kind 
of  information  from  the  right  sources  to  right  people  at  the 
right  time  in  the  right  form.  This  will  require  both  pushing 
information  to  decision-makers  in  the  field  at  various 
echelons,  and  enabling  these  decision-makers  to  pull  the 
information  that  they  deem  relevant.  To  address  this 
problem,  we  must  develop  a  secure  battle  management 
information  system  and  supporting  infrastructure,  and 
refine  human-computer  information  procedures  so  as  to 
manage  and  display  information  in  ways  that  people  can 
understand  and  take  advantage  of  human  intuition.  This 
symposium  focused  on  this  latter  issue.  Specifically,  how 
can  we  better  use  collaboration  and  visualization 
technologies  to  enable  individuals,  teams,  and 
organizations  to  operate  more  effectively? 

Problem  Focus — Mastering  Information:  Command 
and  control  operations  become  increasingly  more 


complex  if  not  chaotic  under  real  battle  conditions.  At 
the  same  time,  the  importance  of  psychosocial  factors 
in  battle  management  operations  increases.  Gen. 
Franquart  (Keynote  III)  asks  how  we  visualize 
information  and  mentally  grasp  it.  His  particular 
concern  is  situations  of  non-conventional  war,  inter¬ 
state  versus  intra-state  conflict  where  the  aim  is  not 
necessarily  victory  but  political  legitimacy  by  violence 
in  all  its  varieties  and  intensities.  In  such  conflicts,  the 
commitment  of  force  has  a  new  purpose.  Politically  at 
the  strategic  level,  force  can  be  used  to  conciliate 
between  opposing  claims  of  legitimacy  or  create  new 
legitimacy.  The  military  objective  is  not  to  attack 
Clausewitzian  centers  of  gravity,  but  to  use  force  to 
"master  violence,"  to  assert  control  on  key  operational 
domains  by  countering  the  expressions,  consequences, 
and  especially  the  causes  of  violence.  This  requires 
information  control — knowledge  of  violence 
perpetrators  and  organizations,  and  curbing  their 
motivations.  In  terms  of  intelligence,  we  have  to 
broaden  our  field  of  action  to  include  the  human  and 
structural  dimensions.  We  must  look  not  only  at  the 
physical  space  occupied  by  a  community  or  ethnic 
group  (valley's,  peaks,  checkpoints),  but  also  at  the  key 
points  in  human  space  (churches,  markets)  and  the 
structural  space  that  the  community  believes  that  it 
needs  to  flourish  (production  and  communications 
facilities).  The  challenge  is  to  develop  visualization 
techniques  that  can  help  us  comprehend  the  different 
types  of  networks  that  exist  in  physical,  human,  and 
structural  space.  But  we  must  go  beyond  synthesis  and 
judgement  to  "master  the  information"  using  human- 
centered  processes  in  both  the  physical  and 
psychological  domains. 

Problem  Recognition — Information  Overload:  Many 
symposium  participants  noted  the  problem  of 
information  overload.  The  amount  of  informed  use  of 
information  by  battlefield  commanders  to  make 
decisions  is  in  practice  quite  small.  With  the  advances 
of  information  technology,  more  information  is 
presented  to  the  decision-maker  than  can  possibly  be 
used.  As  Mr.  Flemisch  and  Dr.  Onken  note  (Paper 
#23),  more  information  and  more  functionality  do  not 
mean  better  usability  or  usefulness;  on  the  contrary,  it 
can  cause  problems  for  operator  information 
processing.  In  practice,  both  tactical-  and  higher-level 
decision-makers  increasingly  place  a  premium  on  the 
ability  to  select  out  information  from  that  presented  to 
them  by  machines  or  command  staff. 

Mr.  Alexander  and  Dr.  Gartner  (Paper  #12)  note  that  in 
the  field,  operators  can  be  overloaded  by  large  amounts 
of  changing  information  that  cannot  be  kept  up  to  date 
manually,  especially  in  times  of  stress.  At  operations 
centers,  computers  process  and  visualize  vast  amounts 
of  information  that  can  still  overload  those  that  have  to 
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interact  with  the  data.  Moreover,  staff  interaction  can 
often  be  limited.  Humans  will  be  influenced  by  the 
information  available  (technical)  and  perceivable  (the  real 
bottleneck).  HF  research  must  support  decision  making  by 
enabling  decision-makers  to  create  a  mental  model  of  the 
battlefield  situation  for  themselves,  thereby  helping  to 
minimize  their  own  information  overload.  This  requires 
displaying  huge  amounts  of  complex  data  in  an 
understandable  and  natural  way. 

According  to  Lt.  Col.  Pugh  (Keynote  II),  we  should  seek 
not  merely  better  information,  but  knowledge,  what  Dr. 
Cook  (Paper  #  2)  describes  as  the  fusion  of  information 
with  prior  knowledge  that  drives  the  process  of 
information  usage  and  allows  for  more  effective  action. 
Pugh  suggests  that  at  present,  effective  knowledge  (that 
is,  action  based  knowledge)  can  only  really  reside  in 
humans.  As  Dr.  Artman  suggests  (Paper  #2),  human 
beings  have  context  sensitivity  and  flexibility  that 
machines  cannot  rival.  Hence,  humans  cannot  be  isolated, 
removed,  or  disengaged  from  the  system. 

Solution  space  must  include  Human  Factors  research: 

If  we  accept  that  knowledge  superiority  is  the  real 
required  capability,  Pugh  argues,  then  it  becomes 
apparent  that  the  processes  we  develop  to  achieve  that 
requirement  must  be  human  centered  rather  than 
technology  centered.  Thus,  research  into  human  factors 
(HF)  will  be  a  key  factor  in  identifying  and  ensuring  the 
eventual  benefits  of  collaboration  and  visualization 
technology  underpinning  knowledge  superiority. 

Pugh  argues  that  we  need  to  develop  a  greater 
understanding  of  HF  as  people  increasingly  become 
integrated  within  the  system.  BG  Alexander  wants  HF 
research  to  focus  not  on  the  ergonomics  of  that  integration 
(such  as  the  preferred  slant  of  a  keyboard),  but  on 
incorporating  the  physiology  of  how  we  think  and  leam 
(using  as  many  of  our  senses  as  possible,  including  sound) 
into  human-machine  interface  design.  We  must,  as  Dr. 
Cook  put  it,  focus  on  the  interface  to  allow  "cavemen”  to 
control  the  system.  Pugh  wants  to  strike  a  balance 
between  HF  and  technology  to  ensure  that  technology 
designed  to  support  command  does  not  have  the  opposite 
effect  (e.g.,  disrupting  manual  processes  that  otherwise 
work  well).  He  urges  exploring  naturalistic  decision¬ 
making  to  gain  a  better  understanding  of  how  individuals 
and  teams  make  decisions  in  complex  environments. 

According  to  Prof.  Vasmes  (Keynote  IV),  HF  research  to 
date  has  missed  the  big  picture — decisions  by  decision¬ 
makers  in  real  battlefield  environments.  Under  conditions 
of  severe  stress  (survival  or  death),  decision-makers 
assess  the  situation  unreliably,  either  coping  or 
defensively  altering  their  perception  of  the  situation  to 
enable  action  (or  inaction),  and  cannot  accurately  project 
the  consequences  of  their  action.  Decision-makers  use 
mental  rules  of  thumb — shortcuts  that  may  prevent  them 


from  obtaining  the  most  accurate  information,  and 
biases  (e.g.,  defense  mechanisms)  that  enable  them  to 
cope.  We  must  therefore  consider  the  perceptual,  the 
cognitive,  and  the  stress  aspects  of  decision-making. 

Pugh  also  believes  HF  research  must  explore  the  use  of 
technology  in  the  command  estimate  process  to  enable 
decision-makers  to  operate  in  a  "knowledge-based" 
environment.  It  is  not  enough  to  focus  just  on  the 
technology.  HF  research  must  play  a  role  in  developing 
not  only  the  requirements  for  technology,  but  also  the 
requirements  for  new  organizational  structures  and  the 
process  of  changing  the  command  culture  so  as  to  get 
the  users  to  use  the  technology  that  is  developed. 
Success  ultimately  will  derive  from  bringing 
technology  into  the  mainstream  of  military  business, 
using  technology  to  help  focus  on  creating  what  others 
call  organizational  knowledge,  a  human-created  product 
of  information,  experiences,  values,  processes  and 
cultures.  Ultimately,  Pugh  cautions,  we  must  recognize 
that  the  potential  offered  from  enhanced  and  tailored 
information  will  never  be  fully  realized. 

In  effect,  Pugh  is  talking  about  using  technology  and 
HF  research  to  promote  knowledge  management,  a 
cultural,  social,  and  human  systems  discipline  as  well  as 
a  technology-enabled  program.  Knowledge 
management  activities  are  corporate  strategies 
employed  to  foster  innovation,  knowledge  transfer, 
improve  business  process,  and  enhanced  organizational 
learning.  There  are  activities  critical  to  knowledge 
creation  and  innovation,  such  as  knowledge  exchange, 
capture,  reuse,  and  internationalization.  There  are  also 
elements  that  enable  or  influence  knowledge-creation 
activities — for  example,  measurement  (e.g., 
performance  and  effectiveness),  policy,  process, 
technology,  and  culture.  The  papers  presented  at  the 
symposium  examined  both  sets  of  activities — the 
processes  of  enabling  or  influencing  knowledge- 
creation,  as  well  as  implementing  knowledge-creation 
activities — for  both  individuals  and  teams,  and  of  the 
system  itself. 

II.  Research  Enabling  Knowledge  Creation 

’’Enabling”  visualization  techniques:  Col.  Louisell 
(Paper  #1)  suggests  that  the  emerging  complexity  of  the 
international  security  environment  requires  a 
networked,  multi- dimensional  approach  to 
understanding  complex  behaviors  quickly.  He  argues 
that  Systems  Dynamics  visualization  models  can  help 
decision-makers  leverage  the  effects  of  networked, 
information-based  warfare.  As  Major  R.  King  (the 
presenter  of  this  paper)  noted,  Systems  Dynamics 
models  start  with  the  assumption  that  there  are  no 
"new"  situations;  the  challenge  is  to  understand  the 
emerging  patters  from  past  situations.  Such  models, 
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King  suggests,  offer  a  quick  solution,  though  not  perhaps 
the  perfect  solution. 

Dr.  Sykora,  Dr.  Dworak,  Mr.  Michalek,  and  Mr.  Novotny 
(Paper  #6)  use  dynamic  sociometry  to  analyze  systems 
based  on  uncertain,  ambiguous,  and  poorly  defined 
elements.  They  focus  on  targeting  decisions  under  NO 
PEACE  /  NO  WAR  conditions.  In  this  method,  the 
relations  of  elements  (subjects,  individuals,  and  groups)  in 
a  complex  system  are  analyzed,  and  the  results  presented 
in  a  map,  where  the  distances  represent  social  relations 
(sympathetic  or  aversive)  and  altitudes  correspond  to  high 
or  low  social  positions.  They  conclude  that  dynamic 
sociometry  is  a  potent  and  reliable  tool  for  modeling 
complex  situations.  It  can  be  used  as  a  predictive  tool  in 
evaluating  the  impact  of  command  decisions  under 
extreme,  poorly  defined,  and  uncertain  situations. 

Dr.  Van  Delft  and  Dr.  Passenier  (Paper  #  14)  focus  on  the 
use  of  multiple  views  of  the  tactical  situation  to  enhance 
situational  awareness  (see  below)  and  improve  situation 
assessment.  Currently,  tactical  command  information 
systems  present  information  in  a  single,  two-dimensional 
’’bird's-eye  view.”  New  advances  in  graphics  capabilities 
and  display  technology  enable  the  application  of  three 
dimensional,  stereoscopic  displays.  Their  research 
suggests  that  higher  information  transfer  rates  (e.g., 
improved  speed  of  detection  for  high-priority  targets)  can 
be  obtained  with  parallel  (related,  but  non-integrated) 
presentation  of  multiple  views  on  the  tactical  situation 
(mixed  2D  and  3D),  with  a  decrease  of  user  interaction 
with  the  system.  In  addition,  using  tactical  objects  in 
multiple  views  enables  the  user  to  modify  and  reconfigure 
the  tactical  workstation  for  effective  supervision  of  and  a 
rapid  response  to  the  tactical  situation  at  hand. 
Stereoscopic  technology  also  supports  the  visual 
separation  of  different  information  layers  (e.g.,  mission 
and  tactics)  with  each  layer  containing  two-dimensional 
representations.  As  a  result,  many  categories  of 
information  can  be  brought  together  in  one  integrated 
graphical  representation  of  the  environment  and  the 
tactical  situation.  The  next  challenge  is  to  develop  a  tool 
for  easy  re-organization  of  information  display  when 
operators  have  to  switch  between  tasks  or  when  changes 
in  the  tactical  situation  take  place.  Van  Delft  and 
Passenier  believe  that  developing  and  testing  an  object- 
oriented  interface  design  enabling  direct  manipulation  of 
tactical  objects  is  the  first  step  to  meet  this  challenge. 

Measuring  Situation  Awareness:  Ms.  Blackwell  and 
Ms.  Redden  (Paper  #7)  examine  the  impact  of  situational 
awareness  (SA)  on  unit  effectiveness  of  dismounted 
infantrymen  engaged  in  Military  Operations  in  Urban 
Terrain  (MOUT)  missions.  They  define  SA  as  the  ability 
to:  (1)  quickly  perceive  and  then  discriminate  between 
facets  of  the  tactical  environment;  (2)  accurately  assess 
and  reassess  the  where,  when,  and  why  of  that 


environment;  (3)  know  and  understand  the  nature  of  the 
tactical  situations;  and  (4)  extrapolate  near  term  courses 
of  action  based  on  this  understanding.  Their  research 
team  first  used  the  Goal-Directed  Knowledge 
Elicitation  Technique  to  solicit  situation- specific,  field 
relevant  mission  needs  from  subject  matter  experts.  The 
resultant  mission  needs  then  formed  the  basis  of 
specific  SA  measures  as  a  means  to  evaluate  the  effect 
of  MOUT  advanced  concept  technology  (e.g.,  giving  all 
soldiers  a  radio)  on  a  unit's  SA.  The  team  then 
developed  an  approach  for  applying  the  SA  measures  to 
experimentation — the  Questionnaire  Assessment  of 
Knowledge  Technique — to  quantify  the  impact  of 
candidate  technologies  on  individual  and  small  unit 
operational  effectiveness.  They  conclude  that  the  freeze 
frame  methodology  (stopping  the  exercise  at  regular 
intervals  to  elicit  participant  responses)  was  particularly 
valuable  in  discriminating  between  baseline  and 
technology  conditions  and  tracking  learning  curves  over 
time. 

Follow  the  path  well  traveled:  According  to  Dr. 
Kirchenbaum  (Paper  #15),  we  often  develop 
battlespace  management  systems  by  decomposing  the 
problem  into  many  functions  and  tasks.  Decomposition 
facilitates  efficient  engineering  of  the  algorithms  and 
programs,  but  the  particular  scheme  used  is  not 
necessarily  congruent  with  the  way  that  the  decision¬ 
maker  solves  the  problem — evaluate  the  available 
information,  predict  the  effects  of  various  action 
options,  and  communicate  the  decision.  Once  the 
problem  has  been  decomposed  and  then  analyzed  with 
the  help  of  information  management  tools  and  decision 
aids,  the  decision-maker  must  put  it  back  together  in  a 
mental  information-fusion  process,  usually  without  the 
help  of  decision  tools.  Kirchenbaum  proposes  starting 
with  knowledge  of  the  decision-maker  and  then 
designing  information  management  decision  aids  that 
support  the  knowledge  schema  and  procedural 
structures  already  used  by  the  expert  decision-maker. 
That  is,  construct  procedural  paths  through  the  task  and 
the  information  that  the  decision-maker  already  uses, 
just  as  how  natural  paths  occur  because  of  repeated  use 
by  pedestrians.  Kirchenbaum  hypothesizes  that  this 
approach  would  lead  to  a  more  efficient  decision 
performance — equal  or  better  performance  in  a  shorter 
time,  with  less  effort.  Her  preliminary  experimental 
results  supported  the  hypothesis:  organizing 
information  based  on  procedural  knowledge  can 
facilitate  performance  of  a  complex,  time-driven  task; 
performance  suffers  in  the  absence  of  such  a 
framework. 

Interface  design:  Three  major  influences  induce 
instability  in  command  and  control  information  systems 
over  time:  rapidly  changing  conditions  regarding 
applied  technology,  changes  in  operational 
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requirements,  and  the  need  to  support  different  user 
profiles.  As  a  consequence,  a  huge  amount  of  information 
and  knowledge  in  different  data  types  has  to  be  managed 
and  processed  in  distributed  communication  networks. 
From  the  Human  Factors  point  of  view,  the  challenge  is  to 
provide  users  direct  and  easy  access  to  the  information 
that  they  actually  need  in  operational  situations.  Cognitive 
automation  and  assistant  systems  offer  promise  to 
handling  the  overwhelming  amount  of  information  battle 
management  systems  will  offer  in  the  future. 

To  cope  with  information  overload,  Mr.  Lenz  and  Dr. 
Ronken  (Paper  #16)  investigated  a  prototype  cognitive 
assistant  system,  the  Crew  Assistant  Military  Aircraft 
(CAMA),  designed  to  enhance  tactical  transport  crews' 
situation  awareness  and  multifunctional  task  handling. 
CAMA  "guides"  rather  than  "directs"  the  crew  in  situation 
monitoring  (perception  and  interpretation),  diagnosis, 
decision-making  and/or  planning,  and  execution.  The 
system  independently  assesses  the  goals  of  the  crew  and 
the  tactical  situation  environment,  detects  possible 
conflicts  of  crew  actions  and  current  plans  given  changes 
in  the  environment,  and  initiates  a  natural,  human-like 
communication  to  warn  the  crew  of  possible  conflicts  and 
propose  corrective  actions.  They  suggest  that  cognitive 
automation  ("guiding"  vice  "directing")  can  improve 
productivity  without  loss  of  safety. 

Mr.  and  Mrs.  Raster  and  Raster  (Paper  #21)  focus  using 
"componentware"  technology  to  create  flexible  system 
architectures  (macro-view)  with  concrete  military 
application,  object-oriented  software  techniques  (micro¬ 
view).  They  argue  that  by  using  this  approach,  designers 
can  develop  highly  flexible  systems  that  can  be  adapted 
easily  to  user  needs  and  task  requirements. 

Drs.  Young,  Eggleston,  and  Whitaker  (Paper  #19) 
investigate  what  functionality  should  be  provided  to  users 
of  a  future  Joint  Battlespace  Infosphere  that  employs 
intelligent  agents  to  seek,  retrieve,  and  fuse  information 
autonomously.  The  authors  believe  that  new  types  of 
direct  manipulation,  work-centered  interfaces  are  needed 
to  reduce  decision  time  and  manning  while  maintaining 
positive  control  over  the  command  and  control  system. 
With  web-based  interface  techniques  and  agents, 
designers  can  transition  the  human  computer  interface 
layer  from  a  mechanism  to  execute  tasks  into  a  decision- 


aid  that  supports  cognitive  information  processing. 

This  involves  allocating  'off-task'  system  domain 
actions  (data  retrieval  and  fusion)  to  agents,  leaving  the 
'on-task’  activities  for  the  on-screen  display  elements 
presented  to  the  decision-makers. 

Information  presentation — testing  assumptions: 

Usability  issues  are  complex  and  require  cognitive 
based  investigations  in  multiple  contexts  and  tasks. 
Gigley  (Paper  #20)  reports  on  research  employing 
interactive  multimedia  and  user-centered  design  to 
enhance  the  decision-maker’s  ability  to  (1)  find 
information,  (2)  query,  refine,  and  process  information, 
and  (3)  make  and  communicate  decisions.  The  study 
reports  that  in  some  situations,  animation  as  a  language 
to  enhance  information  transfer  may  not  be  a  universal 
solution — some  can  learn  from  animation  (especially 
when  the  student  already  understands  and  can  mentally 
visualize  the  concept),  but  for  others  animation  may 
hinder  understanding.  Other  research  suggests  that  the 
utility  of  2-D  and  3-D  presentation  formats  vary  with 
the  task,  but  that  3-D  presentation  can  often  introduce 
ambiguity  of  understanding.  Often,  non-realistic  2D 
icons  are  the  easiest  to  learn  and  to  recognize,  thereby 
enhancing  situation  awareness.  Finally,  other  research 
suggests  virtual  environments  have  applicability  to 
teams  and  integrated  team  tasks,  but  determining 
usability  requires  detailed  evaluation.  While  costly, 
using  the  right  level  of  evaluation  at  the  right  time  in 
the  design-usability  process  can  lead  to  significant  cost 
savings.  The  lesson  from  this  research  is  that  it  is 
important  to  test  our  assumptions,  which  particular 
techniques  will  enhance  or  hinder  information  transfer, 
and  which  will  not.  Cost  saved  from  the  right  kind  of 
evaluation-doing  the  testing  and  evaluation  up  front  to 
aid  design  for  the  task  /  application  /  population  is  not 
only  cost  effective,  but  also  avoids  training  on 
workarounds  or  paying  the  cost  of  a  single  catastrophic 
failure. 

III.  Research  Implementing  Knowledge-Creation 

Framework:  Figure  1  shows  the  process  of  knowledge 
creation  and  effective  use,  using  just  one  among  many 
ways  to  describe  data,  information,  and  knowledge  and 
their  interrelationships. 
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Figure  1:  Framework  for^“useful  and  useable”  information  processing 
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Working  from  the  bottom,  we  (individuals  and  teams, 
together  with  our  machines)  obtain  "data”  by  observing 
and  measuring  physical  processes,  and  then  processing 
those  observations  and  measurements.  To  create 
"information,”  we  index,  organize,  and  store  the  data  for 
subsequent  retrieval.  To  obtain  "knowledge,"  we  must 
both  understand  and  explain  the  information  in  context — 
that  is,  we  must  comprehend  static  and  dynamic 
relationships  between  sets  of  data  and  information,  and 
synthesize  models  to  explain  those  relationships  to 
decision-makers  (i.e.,  ourselves,  our  colleagues,  and 
others  higher  in  the  decision  chain).  Finally,  we  must 
apply  knowledge  to  implement  a  plan  or  action  effectively 
to  achieve  a  desired  goal  or  end  state. 

To  make  data  and  information  useful  and  useable, 
decision-makers  need  to  achieve  and  maintain  Situational 
Awareness  (SA)  (see  earlier  definition).  According  to 
V semes,  there  are  three  SA  levels.  Perception  is  the 
lowest  level  of  SA,  involving  only  the  conscious 
knowledge  that  something  is  present  in  the  environment. 
As  shown  in  Figure  1,  failures  in  perception  correspond  to 
failures  of  observation  and  organization.  Comprehension 
(Level  2  SA)  is  the  synthesis  of  disjointed  perceptions  to 
that  understanding  of  the  significance  of  the  perceptions  is 
present.  Failures  in  comprehension  correspond  to  failures 
in  understanding.  Projection  (Level  3  SA)  is  the  ability  to 
project  future  courses  of  action  based  upon  the 
understanding  gained  from  Level  2  SA.  Failures  in 
projection  correspond  to  failures  in  application.  Vaemes 
reports  that  failures  in  SA  are  the  main  cause  of  aviation 
accidents.  Of  the  SA  failures,  three-quarters  of  accidents 
are  due  to  level  1  failures,  20  percent  by  level  2,  and  only 
4  percent  level  three.  In  other  words,  we  have  to  work 


first  on  perception,  then 
comprehension,  and  then  projection. 

Mr.  Distelmaier,  Mr.  Dorfel,  and  Dr. 
Doring  (Paper  #1 7)  present  a 
slightly  different  concept  structure 
of  human  problem-solving  activities. 
They  also  divide  complex  tasks  into 
three  different  performance 
phases — situation  perception, 
situation  assessment,  and  solution 
generation — that  correspond  roughly 
to  the  vertical  processing  activities 
depicted  in  Figure  1 .  Yet  they  add 
the  complexity  of  differing  cognitive 
task  levels  (skill-,  rule-,  and 
knowledge-based  behavior)  at  each 
activity  level,  with  applicable 
cognitive  tasks  varying  depending 
on  the  situation  encountered  during 
each  activity  level.  Hence,  if 
operators  encounter  routine  situations,  they  use  skill- 
based  information  processing  to  perceive  and  recognize 
the  situation  and  then  take  decisive  action.  If  the 
available  information  does  not  directly  lead  to  a 
reflexive  situation/reaction  routine,  operators  tend  to 
interpret  the  available  information  using  known 
heuristics  and  then  generate  solutions  or  appropriate 
courses  of  actions,  if  necessary.  Finally,  with 
ambiguous  or  uncertain  information  and  unfamiliar  or 
ill-structured  situations,  operators  perform  situation 
diagnosis  using  knowledge-based  problem  solving  and 
evaluation  techniques  before  proceeding  to  action 
planning  and  decision,  as  appropriate. 

Cook  suggests  (Paper  # 2 )  that  the  process  of 
knowledge  development,  creation,  and  maintenance 
itself  drives  the  process  of  information  usage.  Was  that 
process  primarily  individual-  or  team-based?  Team 
knowledge  management  introduces  additional 
complexities;  even  if  knowledge  is  created,  the  system 
must  be  able  to  discern  who  should  have  the  relevant 
information,  in  what  form,  and  when.  Hence,  as  shown 
in  Figure  1 ,  HF  research  must  look  at  ways  to  enhance 
both  individual  and  team  SA  performance. 

Research  into  perception:  Mr.  Flemish  and  Dr.  Onken 
(Paper  #23)  use  a  prototype  eye  measurement  system  to 
analyze  visual  attention  of  pilots,  hypothesizing  that 
missing  visual  attention  is  a  strong  indicator  for  mission 
situation  awareness,  itself  a  strong  contributing  factor 
for  accidents. 

Research  into  comprehension:  Dr.  Essens  (Paper  #3) 
examined  individual  and  team  performance  in  frigate 
operations  centers.  He  discovered  that  individual  and 
team  processes  compete  with  one  another.  Under 
conditions  of  load,  demanding  tasks  (such  as  team 
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processes)  are  dropped  as  individuals  focus  on  tasks  for 
which  they  are  directly  accountable.  Since  high  workload 
conditions  are  normal  in  an  operational  command  center, 
the  question  is  how  to  manage  that  load  more  effectively. 
His  research  suggests  that  managing  workload  requires 
making  organization  processing  capacity  more  flexible 
and  adaptive,  improving  the  quality  of  the  team  through 
training,  and  freeing  team  leaders  from  tasks  of  direct 
production  in  favor  of  management  tasks. 

Essens,  Mr.  Rasker  and  Dr.  Post  (Paper  #4)  describe  a 
research  framework  for  examining  the  effectiveness  of 
command  center  teams.  A  command  center,  they  note,  is 
itself  an  information  processing  system.  Each  team 
member  has  a  specific  role,  expertise,  and  information 
sources,  but  effective  team  action  requires  communication 
and  coordination.  They  describe  a  framework  of  five 
research  methods  to  investigate  command  center  teams  by 
examining  different  parts  of  the  puzzle  on  how  to 
optimize  effective  teamwork.  Modeling  implies  breaking 
up  a  whole  human-human-machine  system  into  its 
essential  elements.  It  is  an  analysis  resulting  in  a  clear 
description  of  the  system  as  a  system.  Modeling  addresses 
the  question  "What  is  the  information  flow  in  the 
command  center?"  Obsewation  is  needed  to  identify 
possible  bottlenecks  in  command  center.  Through 
observation,  we  can  gain  insight  into  the  composite  set  of 
factors  that  influence  command  center  team  effectiveness. 
Experimentation  allows  for  systematic  investigation  of 
single  factors— for  example,  the  effect  of  intra-team 
feedback  on  developing  shared  mental  models.  With 
insight  from  these  three  techniques,  team  design  (or 
redesign)  can  focus  on  improved  effectiveness.  For 
example,  team  design  can  focus  on  the  best  layout  of  the 
command  center  to  encourage  intra-team  feedback  of 
improving  briefing  sessions.  Finally,  any  particular  design 
may  need  an  evaluation  to  determine  how  team 
performance  is  affected.  In  total,  the  research  framework 
represents  an  integral  approach  for  investigating 
command  center  teams. 

Maj.  Worm  (Paper  #9)  reports  on  work  using  the  Action 
Control  Theory  (ACT)  framework  to  examine  the 
dynamics  of  human-machine  systems  in  tactical  mission 
settings  and  scenarios — in  this  case,  time-critical  air 
traffic  control,  process  control,  emergency  response,  and 
military  operations.  ACT  is  a  composite  framework 
encompassing:  cognitive  systems  engineering;  systems 
and  control  theory,  and  cybernetics;  decision  making  in 
complex  command  and  control;  and  psychophysiology. 
Worm  integrated  ACT  with  the  Tactical  Real-time 
Interaction  in  Distributed  EnvironmeNTs  (TRIDENT) 
method  to  assess  workload  and  tactical  performance  in 
battalion-level  battle  command  situations.  He  found 
significant  relations  between  workload,  time  pressure, 
cognitive  complexity,  and  physiological  stress  responses. 
Worm  argues  that  the  ACT/TRIDENT  approach  will 


facilitate:  (1)  identifying  limiting  factors  of  a  specific 
individual,  unity,  system,  procedure,  or  mission;  (2) 
assessing  the  magnitude  of  influence  of  these  factors  on 
overall  tactical  performance;  (3)  proposing  measures  to 
support,  control,  and  improve  insufficient  capabilities 
and  contribute  to  successful  accomplishment  of  future 
missions. 

Maj.  Craciun  (Paper  410)  focuses  on  information 
overload  for  land-based  tactical  electronic  systems. 
Given  improvements  in  electronics  and  computer 
technologies,  electronic  warfare  systems  have 
increasingly  relied  on  computers  to  process  information 
in  a  dense  signal  environment.  Nevertheless,  operators 
can  never  be  completely  eliminated  from  the  system. 
Yet  human  capabilities  have  not  kept  pace  with 
technological  developments  in  the  complex  and  diverse 
processing  tasks  and  roles  for  EW  systems.  Craciun 
argues  that  computers  and  operators  take  different 
approaches  to  information  flows  for  key  tasks.  Consider 
emitter  identification  and  evaluation  and  reporting, 
tasks  that  cannot  be  performed  fully  automatically  and 
need  quick  reaction  times  from  the  human  operators). 
Computers  draw  very  specific  based  on  processing 
pulses  and  signals  using  fixed  format  commands.  By 
contrast,  human  operators  draw  conclusions  based  on 
their  perception  of  the  whole  electronic  environment 
(derived  from  experience  and  indicators),  even  when 
there  is  not  enough  data.  Visualization  technology  and 
techniques  can  reduce  clutter  and  isolating  critical 
signals,  thereby  helping  the  operator  to  focus  only  on 
the  critical  information  needed  for  key  decision  points. 

Dr.  Dudfield,  Mrs.  Macklin,  and  Mr.  Feamley 
(Paper  #13)  describe  research  into  the  use  of  shared 
large  screen  displays  (LSDs)  for  visualizing  the 
battlespace.  They  propose  that  use  of  LSDs,  by 
presenting  more  complete  and  accurate  information, 
can  enhance  situational  awareness  and  increase  the 
probability  of  better  command  team  decision-making. 
Understanding  the  command  team's  information 
requirements  is  crucial  to  gaining  these  advantages.  Just 
as  important,  however,  is  understanding  and  measuring 
how  the  design  of  the  LSD  can  impact  on  Human 
Computer  Interaction  to  best  obtain  the  potential 
performance  benefits  of  team  decision  making  with 
shared  information  space.  Their  research  suggests  that 
LSDs  can  be  particularly  useful  in  briefing  and  team 
situational  awareness  applications.  To  do  so,  LDS  must 
be  flexible  and  re-configurable,  designed  around  how 
the  user  team  works  rather  than  dictating  how  the  user 
team  works.  They  must  support  intra-team 
communication,  but  also  support  individuals  carrying 
out  tasks  independently.  Ideally,  LSDs  should  be 
provided  in  both  fixed  and  deployable  versions  with 
similar  functionality,  and  be  simple  and  straightforward 
to  operate. 
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Dr.  Grau,  Dr.  Hourlier,  and  Prof.  Amalberti  (Paper  #22) 
report  on  their  "Electronic  Copilot"  project,  a  human- 
centered  design  effort  based  on  the  philosophy  of 
cooperative  assistance  to  provide  to  a  pilot  assistance  like 
that  provided  by  another  crewmember.  The  key  is 
understanding  how  humans  approach  the  need  for 
cognitive  compromise — changing  reliance  on  automated 
systems  depending  on  the  situation,  both  in  the  external 
environment  and  with  the  pilot  directly,  and  the  perceived 
level  of  overall  risk.  The  benefits  of  the  electronic  copilot 
are  difficult  to  measure.  Beyond  the  technical  issues  and 
the  challenge  of  integrating  artificial  intelligence 
technology  into  knowledge-based  systems,  their 
principles  of  cooperative  assistance  offer  hope  for 
keeping  humans  in  the  loop  of  complex  system  control. 

Research  into  application:  Dr.  Post  and  Mr.  Hamaker 
(Paper  #5)  propose  a  support  concept  for  staff  planning 
they  call  SmartStaff.  SmartStaff  consists  of  individual 
workplaces,  generation  and  representation  of  ideas,  and 
shared  interactive  large  screen  displays.  They  evaluated 
this  SmartStaff  concept  during  a  simulated  operation  by 
the  Royal  Netherlands  Navy's  Task  Group  Staff.  Based  on 
participant  questionnaires,  they  conclude  that  SmartStaff 
provides  better  general  support  for  group  decision  making 
than  the  current  work  environment.  While  the  quality  of 
the  final  plan  did  not  markedly  improve,  SmartStaff 
supported  better  presentation  and  conveyance  of  ideas, 
facilitated  time  management,  and  decreased  the 
ambiguities  of  the  plans  presented.  Further  research  will 
examine  why  content  did  not  improve,  whether  other 
types  of  teams  may  profit,  and  how  team  planning  is 
actually  carried  out  (e.g.,  satisficing,  or  best  solutions). 

Mr.  Alexander  and  Dr.  Gartner  (Paper  #12)  examine  the 
use  of  the  Electronic  Sandbox — a  virtual  environment 
(VE)  tactical  situation  display  (TSD) — to  support  the 
military  commander  to  process  the  huge  amounts  of 
highly  dynamic  information  provided  by  sensor, 
communication,  and  information  systems.  TSDs  are  used 
in  command  posts  in  the  field  and  at  operation  centers.  In 
theory,  using  VE  as  a  TSD  simplifies  the  interaction  with 
data,  increasing  situational  awareness  and  reducing 
operator  workload.  Alexander  and  Gartner  sought  to  build 
on  the  sandtable  metaphor  by  enabling  dynamic,  real-time 
interaction  and  three-dimensional  changes  of  point  of 
view  by  multiple  operators  working  in  the  virtual  scene. 
While  they  found  the  approach  promising,  further 
research  is  needed  to  discern  sensible  uses  in  differing 
cooperation  environments  varying  the  numbers  of 
operators,  the  time/space  combinations,  and  cooperation 
concepts  (e.g.,  conference  versus  workshop). 

Collaboration  is  of  little  or  no  value  itself  if  it  does  not 
improve  mission  effectiveness.  Mr.  Chapin  and  LCDR 
Dodd  (Paper  #24)  report  on  the  use  of  collaboration 
technology  (text  chat,  voice  audio,  application  sharing, 


whiteboards,  and  web-based  technology)  in  Operation 
Allied  Force  to  support  deliberate  and  crisis  planning 
and  operations.  Collaboration  allowed  United  States 
European  Command's  geographically  separated  sites  to 
work  as  a  team  and  manage  increased  battle 
management  complexity  by  mitigating  the  effects  of 
information  overload,  improving  team  decision-making, 
and  synchronizing  situational  awareness.  Working  with 
NATO  allies  in  a  collaborative  environment  will  likely 
be  an  ever-increasing  operational  requirement.  They 
report  that  performance,  reliability,  and  simplicity  were 
the  primary  factors  that  enabled  collaboration  to  be 
accepted  and  used.  Successful  collaboration  requires  a 
process  owner  with  existing  or  formally  announced, 
delegated  authority  and  recognized  responsibility.  It 
also  requires  a  documented  concept  of  operations. 
Collaboration  does  not  replace  the  need  for  the  right 
combination  of  well-trained,  prepared  personnel  with 
access  to  current  and  accurate  information.  But  the 
operational  lessons  learned  demonstrate  that  judiciously 
applied  to  existing  or  modified  mission  processes, 
collaboration  can  benefit  mission  effectiveness. 

CONCLUSIONS  AND  RECOMMENDATIONS 

The  following  observations  are  based  primarily  (though 
not  entirely)  on  participant  comments  made  during  the 
Capstone  Panel  and  the  open  discussion. 

Should  HF  research  broaden  its  scope?  Many  of  the 
presentations  noted  the  promise  of  teams.  But  how  do 
we  measure  the  effectiveness  of  teams,  and  the  impact 
of  collaborative  and  visualization  technology  and  tools 
on  team  effectiveness?  More  broadly,  Dr.  Dvorjak  asks, 
how  do  measure  the  non-measurable,  non-predictable 
part  of  technical  systems,  the  influence  of  emotions, 
panic,  etc.?  Only  by  going  beyond  the  measurable 
world.  Dr.  Gershon  noted  that  evaluating  the  value  of  a 
phone  conversation,  leadership  styles,  or  interpersonal 
relations  is  very  difficult.  He  proposed  that  those  in  the 
HF  field  thrive  to  integrate  measurable  and  non- 
measurable  (e.g.,  intuitive)  evaluations,  and  therefore 
must  themselves  team  with  others  to  evaluate  team 
effectiveness,  including  (for  example)  designers  or 
artists  or  behavioral  scientists — people  who  understand 
how  things  work  intuitively.  Other  participants 
countered  that  HF  research  is  already  multi¬ 
disciplinary,  sometimes  integrating  people  who 
understand  cognition,  physical  limitations,  and  human 
intuition  with  designers  to  look  at  usability  calculations. 
For  example,  examining  the  process  of  learning  through 
storytelling  is  already  a  standard  process  used  in  design 
development.  The  HF  field,  as  Dr.  Essens  put  it,  is  a 
mixed  bag,  involving  several  different  disciplines,  some 
early  on  in  the  design  phase  of  information 
management  systems,  and  others  at  the  end  of  the 
process  (testing  the  system  after  completion).  Most 
agreed  that  multi-disciplinary  design  teams  (whether 


HF-trained  people  are  in  the  minority  or  the  majority) 
offer  the  best  approach. 

Gen.  Franquart  argued  that  the  problem  of  decision  is 
very  different  according  to  the  level  of  decision  and  the 
level  of  responsibility  in  the  decision.  Tactical  decision, 
for  example,  are  very  different  that  strategic  or  political 
decisions.  Staff-level  decisions  (often  team-centered)  are 
very  different  that  command-level  decisions  (usually 
unitary  actors).  HF  research  can  help  analyze  the  problem 
from  different  levels  of  responsibility  and  action.  Lt.  Col. 
Pugh  agrees,  noting  that  in  combat  at  the  tactical  level, 
operators  are  concerned  primarily  with  what  is  happening 
immediately  around  them.  In  such  environments, 
information  is  almost  peripheral,  interfering  with  thought 
processes. 

Dr.  Leger  suggests  that  HF  research  has  relied  too  much 
on  the  ability  of  decision-makers  to  make  good  decisions. 
We  have  to  prepare  for  unpredictability.  Dr.  Boff  agrees, 
noting  that  the  operator  is  not  a  perfect  receiver  of 
information.  We  should  expect  that  decision-makers  are 
not  going  to  perceive  information  in  the  way  that  we 
designers  might  expect.  In  the  end,  battlefield  information 
systems  must  match  the  logic  of  the  operator.  Hence,  we 
must  design  in  adaptive  interfaces  to  allow  the  operator  to 
use  his  or  her  own  strategies. 

To  simulate  or  not  to  simulate?  Prof.  Vsemes  (Keynote 
IV)  argues  that  the  science  of  information  processing  has 
become  too  segmented  along  the  line  of  specific 
experimental  tasks.  Because  little  information  is 
exchanged  among  research  groups,  the  science  has 
produced  a  lot  of  information  but  contributed  little  to  our 
understanding  of  how  humans  function  in  real- world 
settings.  Human  information  processing  is  a  complex 
system  that  will  never  be  understood  as  a  sum  of  its 
component  parts.  Human  factor  specialists 
must  widen  their  perspective  beyond  the 
simple  tasks  that  dominate  current  research. 

To  analyze  the  decision  support 
considerations  of  the  usability  of 
information  in  battle  management 
operations,  we  must  both  conduct 
laboratory  simulation  experiments  and 
collect  valid  information  from  the  "rear 
battlefield  environment. 

While  some  participants  agreed  that 
research  should  look  at  real  combat 
environments,  others  cautioned  that  the 
value  of  simulations  should  not  be  over¬ 
looked.  Lt.  Col.  Pugh  emphasized  that  we 
have  to  have  simulation  to  be  able  to 
predict  in  advance  what  works,  to  provide 
enough  foundation  and  theoretical  support 
to  mold  new  technology  to  meet 
operational  needs,  and  get  it  right  the  first 


time.  A  spiral  research  agenda  may  be  best,  alternating 
simulation  and  combat  experiments  using  observation, 
simulation,  and  combat  experimentation  where 
appropriate,  cost-effective,  and  applicable. 

Part  of  the  debate  stems  from  the  financial  pressures  to 
create  "demonstrators" — that  is,  prototypes  of  limited 
capabilities  designed  to  convey  principles  and 
applications  Dr.  Essens  referred  to  this  as  the  effort  to 
make  artifacts  work  to  get  money  (research  artifacts) 
rather  than  focusing  on  real  issues.  But,  as  many  noted, 
the  fact  of  life  is  that  the  needs  of  customers,  including 
their  learning  needs  (e.g.,  disbelieving  pen  and  paper 
studies  or  proposals  they  cannot  touch  and  feel),  will 
continue  to  drive  research. 

On  visualization:  Dr.  Eggleston  believes  that  based  on 
the  research  presented,  some  progress  has  been  made 
on  visualization  from  a  HF  perspective.  Yet  we  still 
have  no  consensus  in  the  HF  discipline  on  how  to 
design,  build,  and  evaluate  visualization.  Dr.  Essens 
suggests  central  to  the  development  of  visualization 
from  a  HF  perspective  is  understanding  the  task  and  the 
task  environment.  A  lot  of  the  HF  visualization  research 
relates  to  individual  interfaces,  not  to  developing  a 
common  operational  picture.  Dr.  Boff  suggests  that  this 
lack  of  attention  was  a  significant  hole  in  the 
symposium. 

On  collaboration:  Figure  2  presents  the  collaboration 
space,  showing  different  combinations  of  time  and 
space;  from  synchronous  (same  time)  and  persistent 
(same  place)  in  the  upper  left,  through  asynchronous 
and  non-persistent  unpredictable  collaboration  in  the 
bottom  right.  Illustrative  collaborative  tools  and 
techniques  are  provided  for  each  space/time 
combination  (in  Italics). 


Figure  2:  The  collaboration  space 
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As  indicated,  we  have  suggested  in  which  combinations 
the  twenty-six  symposium  contributions  operate.  All  the 
keynote  addresses  and  three  of  the  papers  operate 
throughout  the  collaboration  space,  and  two  papers 
concentrate  on  asynchronous,  non-persistent 
combinations.  The  largest  number  of  contributions  fit  into 
the  same  time/place  combination.  Yet  in  the  real 
battlefield  environment,  collaboration  (between  human 
and  machine,  individuals,  and  teams)  takes  place  in  forms 
that  span  geographic,  temporal,  and  organizational 
boundaries.  And  in  conditions  of  stress,  persistence  of 
information  may  be  required  to  effect  information 
transmission.  This  suggests  that  HF  research  should 
expand  its  coverage  throughout  the  collaboration  space,  to 
include  cross-cultural  collaboration  (a  topic  absent  from 
this  symposium). 

The  challenge  of  institutionalization:  The  nature  of 
collaboration  systems  as  a  "disruptive  technology" 
(requiring  changed  operational  procedures  and  practices) 
means  that  broad  high-level  management  support  is  a 
necessary  condition  for  success.  Further,  collaboration 
systems  have  to  be  "institutionalized" — that  is,  they  must 
become  part  of  legitimate  and  ongoing  practice,  affect 
structures  and  patterns  throughout  the  organization,  and 
be  supported  by  other  aspects  of  the  system  within  the 
organization’s  domain.  Because  efforts  approved  at  the 
top  can  be  frustrated  by  peripheral  and  lower-level 
participants  within  an  organization,  the  system  of  rewards 
also  has  to  change  to  encourage  the  building  of  coalitions 
and  teams  to  support  and  implement  the  new  tasks.  The 
supporting  network  communications  architecture  also 
must  be  well  prepared  for  deployment  of  a  collaboration 
application. 

The  most  difficult  challenge  is  that  of  creating  a  culture  of 
collaboration — creating  an  organizational  culture  and 
organizational  readiness  to  change  to  support 
collaborative  operations  culture  of  collaboration.  Working 
smarter  means  working  together,  not  harder,  and  breaking 
down  stovepipe  processes  in  favor  of  more  non-linear  and 
agile  decision-making  models.  The  impact  can  be 
dramatic,  as  exemplified  during  the  Kosovo  operation 
when  the  Collaborative  Contingency  Targeting  system 
enabled  the  Cruise  Missile  Support  Activity  and  theater 
commanders  to  reduce  the  timelines  for  distributed 
targeting  for  Tomahawk  Land  Attack  Missiles  from  days 
to  hours.  While  costly,  Mr.  Chapin  and  LCDR  Dodd 
suggest,  a  distributed  information  architecture  may  help 
ensure  a  timely,  appropriate,  and  effective  response.  HF 
must  work  on  measuring  the  value  of  such  architectures  to 
aid  in  system  design. 

Finally  and  most  importantly,  collaborative  and 
visualization  systems  must  be  used  if  they  are  to  reap 
benefits.  This  means  that  intuitive  interfaces  are  needed 
(to  the  extent  possible),  and  adequate  (and  repeated) 


training  programs  must  be  implemented  throughout  the 
organization.  People  revert  to  that  with  which  they  are 
familiar  when  they  are  not  comfortable  with  the  new. 
This  is  especially  true  under  conditions  of  stress. 
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SUMMARY 

Commanders  have  always  faced  significant 
challenges  in  visualizing  the  battlespace;  how  well  they 
have  succeeded  has  determined  their  ability  to  shape 
the  battle  to  their  advantage.  As  our  weapons 
technology  advances  and  shortens  decision  times,  it  is 
imperative  that  our  warfighters  be  able  to  visualize  the 
operational  situation  and  to  share  this  picture  up  and 
down  echelons  and  among  coalition  partners.  We  can 
take  several  steps  to  improve  our  ability  to  pass  a 
common  relevant  operational  picture  of  the  threat 
among  our  various  components,  coalition  partners  and 
national  leaders  that  is  vital  to  synchronize  combat 
operations. 

THE  PROBLEM 

The  inability  to  clearly  visualize  the  friendly 
and  enemy  battlefield  situations  is  a  major  problem 
facing  Warfighters  and  their  staffs.  There  are  four 
major  reasons  we  have  not  solved  this  problem.  First, 
we  have  a  limited  ability  to  rapidly  process,  exploit, 
analyze,  and  fuse  the  large  amounts  of  information 
available  from  the  many  sources  providing  input. 
Second,  we  lack  the  tools  needed  to  help  us 
comprehend  that  information.  Third,  there  is  a 
shortage  in  numbers  and  types  of  sensors  and  systems 
needed  to  display  accurately  and  automatically  red- 
force  and  blue-force  information.  Fourth,  limited 
tactical  communications  hinder  our  ability  to  rapidly 
share  information  across  the  battlefield  in  a 
synchronized  manner  among  all  our  command  and 
control  elements.  This  last  issue  recognizes  the  many 
difficulties  involved  in  passing  information  among  the 
various  echelons,  from  national  through  tactical  levels, 
among  various  functional  disciplines,  and  perhaps  the 


biggest  problem,  and  one  of  particular  interest  to  this 
group,  passing  information  among  coalition  members. 

These  problems  exist  for  a  number  of  reasons. 
It  is  difficult  to  synchronize  the  various  acquisition 
efforts  to  best  leverage  advances  in  computer 
technology.  Each  of  our  Services  approach 
information  processing  and  battlespace  awareness 
differently.  Visualization  requirements  vary  from 
nation  to  nation  within  our  ever-changing  Coalition. 
While  a  number  of  tools  are  available  to  help  us 
understand  this  information,  they  are  not  always 
compatible  with  the  many  types  of  software  and 
hardware  obtained  through  various  acquisition 
programs.  As  a  consequence,  we  often  purchase 
Service-specific,  but  non-interoperable,  software  and 
hardware.  While  this  helps  us  within  our  Services  and 
within  each  of  our  nations,  it  greatly  complicates  our 
ability  to  operate  cohesively  within  a  multi-national, 
joint  alliance. 

Budget  limitations  constrain  the  number  of 
sensors  we  procure,  often  resulting  in  an  incomplete, 
and  inaccurate  picture  of  the  battlefield. 
Communication  among  mobile  units  is  difficult  and 
usually  constrained  by  bandwidth.  Since  many  of  our 
tactical  operations  command  posts  adapt  their 
information  displays  and  management  to  meet  specific 
requirements,  we  often  have  a  problem  formatting 
information  to  meet  the  specific  needs  of  subordinate 
elements  and  functional  areas. 

An  associated  problem  within  communications 
is  multi-level  security.  This  problem  has  plagued  all  of 
us.  Each  different  security  classification  (Top  Secret, 
Secret  one  nation  only,  Secret  releasable  to  our  allies, 
or  in  this  case,  NATO)  requires  different  handling 
instructions.  These  multiple  security  requirements  and 
the  manual  tasks  required  processing  them  cause 
additional  delays  in  passing  information  among 
organizations  and  echelons. 


Paper  presented  at  the  RTO  HFM  Symposium  on  “Usability  of  Information  in  Battle  Management 
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THE  RESULT 

The  problems  outlined  above  produce  several 
significant  consequences.  The  most  obvious  is 
information  often  arrives  late  and  is  outdated  when  it  is 
received.  However,  since  it  is  usually  the  best 
information  available,  it  is  normally  integrated  into  the 
current  intelligence  picture.  Since  this  may  occur  at 
various  times  at  the  various  echelons  and  functional 
areas,  it  results  in  an  unsynchronized,  inaccurate 
common  operational  picture  across  the  battlefield. 
This  creates  problems  synchronizing  information  as 
each  command  post  may  have  a  slightly  different  view. 
Because  we  lack  the  ability  to  pass  a  common  relevant 
operational  picture  of  the  threat  among  our  various 
components  (Air  Force,  Navy,  Army,  and  Marine 
Corps)  and  to  the  joint  and  national  levels,  our  ability 
to  synchronize  combat  operations  is  less  than  efficient. 

REAL  WORLD  EXAMPLE 

At  Headquarters,  United  States  Central 
Command,  we  have  a  number  of  automated 
capabilities,  but  we  do  not  have  the  ability  to  share  a 
real  time  intelligence  picture  with  our  national 
agencies,  Joint  Staff,  subordinate  commands,  and 
allies.  We  maintain  our  common  picture  on  a  map 
board,  and  then  transfer  this  onto  Power  Point 
presentation  slides  that  are  emailed  to  all  concerned. 
The  most  significant  problem  with  this  is  the  timeliness 
of  the  information  being  passed.  It  takes  time  to  create 
and  mail  the  presentation,  ensuring  much  of  the  data  is 
outdated  immediately.  At  the  same  time,  those  to 
whom  we  are  mailing  our  slides  are  probably  getting 
updated  textual  information  before  they  get  our  slides. 
The  differences  between  the  real-time  reporting  and  the 
dated  presentation  slides  cause  confusion.  Video  tele¬ 
conferences  help  synchronize  our  efforts  during  crisis, 
but  these  conferences  consume  time  and  are  not 
efficient. 

IMPACT 

There  is  another  much  more  serious,  but  less 
visible,  consequence  of  the  problem.  Our  lack  of  an 
accurate  and  timely  common  operational  picture 
negatively  impacts  every  step  of  the  intelligence  cycle. 
The  intelligence  cycle  should  not  be  viewed  as  a 
sequential  system,  but  rather  as  a  series  of  steps,  all 
inter-related,  and  all  happening  simultaneously.  It  goes 
on  at  every  echelon;  each  echelon  depends  on  input 


from  higher,  lower  and  adjacent  elements.  Finally,  and 
perhaps  most  importantly,  this  cycle  must  be  linked  to 
operations.  Operational  decisions  not  closely  linked  to 
the  intelligence  situation  are  doomed  to  failure.  First,  I 
will  briefly  discuss  some  of  the  steps  of  the  intelligence 
cycles  and  present  some  thoughts  on  what  we  can  do  to 
fix  this  problem.  Then  I  will  discuss  battlespace 
visualization  requirements  and  some  of  our  thoughts  on 
solutions. 

PLANNING 

In  order  to  plan  among  our  echelons, 
components,  and  allies,  we  must  have  a  common 
picture  of  the  battlespace  and  a  real-time  collaborative 
ability  to  link  our  planning  elements  together  to 
develop  and  synchronize  a  plan.  We  must  have  the 
ability  to  understand  and  comprehend  the  information 
displayed  to  us.  Based  on  that  comprehension,  we 
must  also  have  the  ability  to  develop  future  courses  of 
action,  branches,  sequels  and  other  possibilities  upon 
which  we  can  base  our  plans. 

COLLECTION  MANAGEMENT 

One  of  the  biggest  challenges  we  face  is 
managing  our  national,  theater  and  tactical  collection 
assets,  because  we  lack  the  dynamic  collaborative 
ability  to  display  what  assets  at  each  level  are  doing  or 
are  going  to  do.  This  process  is  even  more 
complicated  when  we  attempt  to  synchronize 
collection  assets  with  our  allies.  The  results  are 
obvious;  we  waste  a  lot  of  effort  duplicating  collection 
missions,  exploiting  and  reporting  similar  activities  on 
the  battlefield,  while  simultaneously  missing 
opportunities  to  collect  on  events  that  may  have  had 
equal  or  greater  importance.  We  need  to  show  what 
will  be  collected  from  various  assets  so  we  can  trade¬ 
off  between  platforms  for  the  most  efficient  collection 
deck.  We  also  need  the  ability  to  predict  what  we  can 
collect. 

EXPLOITATION 

Many  of  the  exploitation  problems  stem  from 
the  collection  management  problem  I  just  described. 
But  even  if  we  fixed  the  collection  management 
problem,  we  still  do  not  have  the  ability  today  to 
consistently  focus  our  exploitation  capabilities  on  the 
most  pressing  problems  first.  Nor  do  we  have  the 
ability  to  easily  collaborate  among  exploitation  centers 
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to  optimize  our  efforts.  Finally,  the  outputs  from  our 
exploitation  centers  are  often  reports  that  are  very 
difficult  to  automatically  integrate  into  the  common 
relevant  operational  picture.  Collaboration  among 
collection  managers,  analysts  and  all  exploiters  would 
significantly  improve  our  exploitation  shortfalls. 

ANALYSIS 

Perhaps  the  group  most  impacted  by  these 
problems  is  our  analysts.  Because  they  cannot  see  the 
plan  or  the  collection  management  picture;  because 
they  rarely  have  the  insight  of  the  exploitation  center; 
and  because  they  often  receive  the  information  after  a 
time  delay,  their  already  incredibly  complex  mission  to 
integrate  information  into  a  common  picture  is  made 
almost  unattainable.  These  analysts  need  the  ability  to 
collaborate  and  view  all  the  steps  of  the  intelligence 
cycle  as  it  progresses.  They  must  be  able  to 
automatically  sort,  parse  and  display  information  in 
various  text  and  graphic  ways,  including  electronic 
two-dimensional  and  three-dimensional  displays. 
Finally,  because  their  output  must  support  a  host  of 
consumers,  these  analysts  must  be  fully  synchronized 
with  operations. 

DISSEMINATION 

Finally,  we  must  be  able  to  disseminate  this 
information  amongst  all  consumers  in  a  timely  manner. 
We  must  be  able  to  quickly  pass  it  to  multiple  echelons 
and  among  our  allies,  often  with  differing  security 
requirements,  in  to  support  decision-making  for  all. 
This  would  be  further  enhanced  through  web-based 
technologies  and  other  available  collaborative  tools. 

TO  FIX  THE  PROBLEM 

Now,  some  thoughts  on  how  we  fix  this 
problem  that  hopefully  will  be  useful  in  stimulating 
further  dialogue.  First  we  have  to  develop  and  evolve 
a  common,  relevant  operational  picture  for  operations 
and  intelligence  that  can  be  used  by  all  operators, 
collection  managers,  exploiters  and  analysts.  Not 
everyone  needs  to  look  at  the  same  picture  at  the  same 
time;  however,  everyone  must  have  the  same 
information,  which  can  be  tailored  and  displayed  to 
best  support  their  requirements  to  accomplish  their  job 
or  function.  To  develop  this  picture,  we  have  to  make 
our  data  feeds  and  systems  interoperable  on  a  wide 
area  network  all  can  use.  This  network  must  have 


different  security  layers  to  protect  information,  similar 
to  the  ways  the  commercial  market  is  securing  the 
Internet  for  business  today.  We  also  need  different 
types  of  displays  at  various  echelons  to  meet  user 
needs.  Some  will  need  large-screen  displays  for  staff 
planning,  execution  and  operations.  Others  will  need 
individual  displays  to  conduct  their  business.  All  need 
access  to  timely  and  accurate  information.  Finally,  all 
must  have  access  to  tools  that  help  them  manage  and 
display  the  information  in  ways  that  they  can 
understand,  so  that  they  may  efficiently  react  to 
dynamic  environments. 

VISUALIZATION 

REQUIREMENTS 

One  of  the  big  challenges  we  all  face  is 
articulating  our  visualization  requirements  in  a  manner 
that  leads  to  the  best  solution.  We  need  the  ability  to 
bring  in  all  types  of  data  and  to  actively  query 
databases  for  additional  information  when  needed. 
This  must  include  all  available  forms  of  information. 
We  need  different  types  of  displays  and  different  types 
of  backgrounds  to  facilitate  comprehension  and 
decision-making.  Ideally,  these  displays  and  the 
information  provided  would  also  be  common  among 
our  various  coalition  partners.  We  also  must  ensure 
that  data  is  provided  the  appropriate  levels  of  security, 
while  still  ensuring  everyone  is  efficiently  connected. 

SOME  THOUGHTS 

Modem  technology  is  only  half  the  answer. 
To  make  all  that  I  have  discussed  a  reality,  we  also 
have  to  have  a  common  set  of  tactics,  techniques  and 
procedures  for  manipulating  and  using  this 
information,  and  we  must  train  together.  Once  we 
develop  these  procedures  and  train  our  people,  we  must 
then  exercise  together,  and  evolve  the  way  we  think. 

CONCLUSION 

Visualizing  the  battlespace  is  indeed  a  very 
difficult  problem.  Commanders  who  have  had  the  best 
understanding  have  always  done  the  best  in  conflict.  It 
behooves  us  to  provide  our  warfighters  with  the  best 
possible  visualization  capabilities. 
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Network-Centric  Warfare 

Key  US  proponents  of  the  Revolution  in  Military 
Affairs  described  future  war  as  a  system  of  systems 
in  which  dominant  battlespace  knowledge  would 
enable  a  system  of  sensors  and  shooters  to  be 
connected  for  the  purpose  of  engagement  through 
an  advanced,  information  technology-based 
command  and  control  function  (Fig  1).  Through 
dominant  battlespace  knowledge,  the  command  and 
control  function  would  achieve  efficiency  levels 
which  would  greatly  alter  the  nature  of  conflict  - 
current  time  constants  in  the  decision,  action, 
feedback  loop  would  be  drastically  shortened.  The 
nature  of  weapons  and  platforms  would  change 
and  the  organization  and  training  of  forces  would 
change. 

Stuart  Johnson  and  Martin  Libicki,  in  their  National 
Defense  University  publication  Dominant 
Battlespace  Knowledge,  offer  an  analytical  tool  to 
understand  the  value  of  dominant  battlespace 


knowledge  in  a  network-centric  force  application 
concept  (Fig  2). 

Perfect  command  and  control  (C2)  is  achievable 
only  with  perfect  information  -  dominant 
battlespace  knowledge.  Dominant  battlespace 
knowledge  essentially  represents  the  command  and 
control  function’s  understanding  of  the  situation  as 
near-perfect.  Although  this  level  of  understanding 
may  be  achievable  in  certain  environments,  its 
potential  is  challenged  by  the  emerging  nature  of 
conflict  -  one  described  by  Samuel  Huntington  as 
the  Clash  of  Civilizations.  Huntington  sees  future 
conflict  as  humanistic,  driven  by  fault  lines  between 
cultures  and  economies.  Conflict  of  this  nature 
challenges  the  ability  to  achieve  dominant 
battlespace  knowledge.  Furthermore,  the  centers  of 
gravity  in  such  a  conflict  are  much  more  broadly 
distributed  across  a  nation’s  economy, 
infrastructure,  international  relationships,  internal 
divisions  and  armed  forces. 


Objects 


Figure  1:  System  of  Systems  Approach 


Paper  presented  at  the  RTO  HFM  Symposium  on  “Usability  of  Information  in  Battle  Management 
Operations”,  held  in  Oslo ,  Norway,  10-13  April  2000,  and  published  in  RTO  MP-57. 
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Potential  Value  of  DBK 


Figure  2:  Dominant  Battlespace  Knowledge  (DBK)  in  Command  and  Control  (C2) 


Changing  Face  of  National  Power 

At  the  same  time  the  nature  of  conflict  is  changing, 
the  nature  of  national  and  Alliance  power  is 
changing.  The  information  age  creates  the  potential 
for  a  seamless  system  of  power  resources  which 
spans  a  spectrum  from  the  strategic  use  of 
information  and  information  operations  to  the 
application  of  information-based  strategic  and 
tactical  weapons  systems.  In  the  future,  a  multi¬ 
dimensional  approach  will  emerge  in  which  ail 
elements  of  national  power  including  economic 
instruments,  political  instruments,  information 
instruments,  and  military  missions  are  applied 


synergistically  across  an  info-kinetic  power 
spectrum  (Fig  3). 

This  will  have  a  significant  impact  on  military 
actions  at  the  operational  level.  No  longer  will  the 
task  force  commander  be  purely  concerned  with  the 
impact  of  his  actions  on  the  enemy’s  forces.  Nor 
will  the  destruction  of  forces  and  the  seizure  of 
strategic  resources  be  the  singular  determinant  in 
changing  enemy  behavior.  The  task  force 
commander  will,  to  a  larger  degree  than  ever 
before,  have  to  integrate  actions  and  the  actions  of 
others  into  a  continuous,  coherent,  multi¬ 
dimensional  plan. 


Figure  3:  Info-Kinetic  Power  Spectrum 
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Complexity  at  the  Operational 
Level 

The  operational  decision  environment  will  become 
non-linear  with  non-linearity  defined  as  a  condition 
in  which  a  system  disobeys  principles  of 
proportionality  and  additivity.  Non-linear  systems 
provide  erratic  or  “chaotic”  responses  to  inputs  - 
cause  and  effect  are  difficult  to  track  through  a 
series  of  nodes  in  which  forces  interact.  Achieving 
a  sufficient  level  of  understanding  in  these  non¬ 
linear  enviroments  will  require  special  tools  and 
familiarity  with  complex  decision  techniques. 

Decisions  in  Environments  of 
Complexity 

Study  of  decisions  in  complex  scenarios  reveals 
two  fundamental  styles  —  an  Analytic  Style  and  a 
Naturalistic  or  Recognitive  Style.  The  analytic 
style  is  characterized  by  a  systematic  data  collection 


effort  and  a  formal  analysis  and  evaluation  of 
various  options.  The  naturalistic  or  recognitive 
style  is  characterized  by  an  intuitive  use  of  patterns 
leading  to  decision  based  on  experience  in  like  or 
similar  situations  (Van  Riper  and  Hoffman,  1997). 
Experts  in  dynamic  decision  environments  —  the 
master  chess  player,  the  NBA  basketball  player,  the 
great  general  —  make  decisions  intuitively  based  on 
the  comparison  of  current  situation  understanding 
to  past  situations  and  outcomes.  These  experts  take 
advantage  of  the  speed  of  the  naturalistic  or 
recognitive  approach  to  act  quickly  -  to 
outmaneuver  their  opponents  by  reducing  the  time 
to  determine  the  action  required.  (Trotter,  1986) 

System  Evaluation 

Understanding  the  complex  causal  chains  in  a  non¬ 
linear  system  requires  the  use  of  one  of  a  limited 
number  of  powerful  system  evaluation  techniques. 
One  such  technique  is  Systems  Dynamics  (Inset  1). 
It  was  developed  by  an  MIT  professor,  Jay 
Forrester.  Forrester  modeled  webs  of  activity  that 


Systems  Dynamics  -  A  Snapshot 

Prior  to  modeling  a  System  of  Alliance  power,  it  is  necessary  to  establish  a  fundamental 
understanding  of  the  Systems  Dynamics  method.  As  inferred  in  the  previous  section,  the 
Systems  Dynamics  approach  to  system  evaluation  is  based  on  the  principle  of  causal  chains 
made  up  of  paired  variables  that  are  related  through  physical  or  information  flows.  In  the 
relationship,  an  independent  variable  acts  upon  the  other  variable,  the  dependent  variable. 
Variables  are  paired  in  graphic  form  to  show  the  direction  of  the  influence  and  the  polarity 
of  the  relationship.  The  key  variables  of  interest  in  a  Systems  Dynamics  model  are  called 
level  variables.  Levels  are  deteriorated  or  increased  by  rate  variables.  The  relationship 
between  level  and  rate  variables  is  depicted  by  a  solid  line  with  an  arrow  head  indicating  the 
direction  of  influence  and  a  plus  or  minus  sign  to  indicate  the  polarity  of  the  relationship.  In 
a  positive  relationship,  both  variables  move  in  the  same  direction.  In  a  negative  relationship 
the  dependent  variable  moves  opposite  the  direction  of  the  independent  variable. 

Inherent  in  the  evaluation  of  a  Systems  Dynamics  model  is  the  identification  of  feedback  loops  in 
which  causal  streams  and  the  information  relationship  within  those  streams  cause  the  system 
to  exhibit  characteristic  behaviors.  First  and  Second  order  positive  feedback  loops  grow  or 
decay  exponentially  once  moved  from  an  equilibrium  state.  First  order  negative  feedback 
loops  correct  from  a  deviation  to  an  equilibrium  state.  Second  order  negative  feedback  loops 
oscillate  around  an  equilibrium  value.  Identifying  feedback  structures  within  a  system  or 
sector  and  understanding  the  behavior  associated  with  structures  present,  will  allow 
prediction  of  behaviors  and  the  evaluation  decisions  which  policy  makers  make  in  an  effort 
to  try  to  influence  a  system’s  behavior.(Drew,  1998).  This  description  is  not  an  all  inclusive 
primer  on  Systems  Dynamics.  It  is  designed  only  to  allow  the  reader  to  evaluate  and 
appreciate  the  model  of  Alliance  power  which  will  be  developed  and  explored  in  subsequent 
sections. 


nset  1 :  Systems  Dynamics  Methodology  for  Depicting  Causal  Streams 
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were  interrelated  through  shared  information, 
shared  physical  resources,  and  interactive  feedback 
loops.  He  demonstrated  system  characteristics 
between  related  variables  and  was  able  to  develop 
causal  chains  that  could  be  evaluated  numerically 
or  graphically  to  provide  insight  to  complex  and 
counterintuitive  behavior.  He  did  this  for  a  world 
model,  for  commercial  industries,  and  for 
government  evaluation  of  policies.  Forrester 
identified  that  systems  are  made  up  of  basic 
feedback  structures  which  have  known  behavioral 
characteristics.  He  demonstrated  how  the  elements 
of  system  structure  and  behavior  could  be  used  to 
understand  the  behavior  of  large,  multi-discipline, 
complex  systems  of  interaction  which  he  terms  as 
metaproblems  (Martin,  1996)  (Drew,  1998). 

System  of  Alliance  Power 

In  order  for  military  commanders  to  develop  a 
systems  view  of  national  and  Alliance  power,  we 
need  a  model  that  links  together  the  various  sectors 
of  influence  of  the  elements  of  power  -  economic 
policy,  political  policy,  information  policy,  and 
deployment  and  application  of  military  force. 


Economic  Sector 

The  economic  sector,  contained  in  the  total  power 
model,  is  represented  by  the  relationship  between 
Capital  Investment,  Capital,  Industry  Output,  and 
Gross  National  Product  (Fig  4).  Note  the  positive 
feedback  loop  which  is  formed  by  this  causal 
stream.  This  positive  feedback  loop,  along  with 
others  identified  in  the  successive  sectors,  are 
points  of  leverage  for  policy  application. 

Military  Attrition  Sector 

A  military  attrition  model  is  included  in  the 
integrated  national  security  model.  It  represents  a 
classic  force-on-force  battle  in  which  force  attrition 
is  the  determinant  of  victory.  The  attrition  model  is 
a  second  order  positive  feedback  loop  —  once 
influenced  in  the  proper  direction  through  an 
overwhelming  initial  attack,  it  can  be  pushed, 
especially  if  time  is  compressed,  reducing  the 
potential  for  restocking  through  domestic  or  foreign 
production.  This  positive  feedback  loop  is  central 
to  evaluating  force  application  policies. 
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Figure  4:  Positive  Feedback  Loop  in  the  Economic  Sector 


Figure  5:  Positive  Feedback  Loop  in  the  Military  Attrition  Sector 
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Political  Power  Attrition  Sector 

The  political  power  attrition  sector  establishes  the 
linkage  between  the  power  of  the  political 
leadership,  the  will  of  the  people,  and  the 
perception  of  the  international  community  (Fig  6). 
This  sector  model  is  in  draft  form  and,  is  subject  to 
refinement.  In  its  current  form,  it  is  a  conceptual 
tool  to  understand  the  causal  relationships  between 
the  factors  that  work  to  provide  a  political  power 
base. 

This  sector  is  central  to  understanding  the  effect  of 
actions  on  the  key  measure  of  effectiveness  -  the 
willingness  of  the  enemy  to  continue  the  conflict. 
Targeting  the  enemy’s  will  is  not  new,  it  was  part  of 
the  total  strategy  employed  by  the  ancient  Chinese 
warrior,  Sun  Tzu.  What  is  new  is  that  information 
combined  with  rapidly  applied,  precise  force 
introduces  an  ability  to  shock  the  enemy  and  cause 
an  early  question  in  the  minds  of  the  people 
(U 11  man  and  Wade,  1996).  The  keys  in  this  sector 
are  the  linkages  between  the  International 
Perception,  Political  Power  of  the  Leadership,  and 
the  Will  of  the  People. 

Major  influencing  factors  from  the  other  two 
sectors  are  triggers  to  set  in  motion  the  two  second 
order  positive  feedback  loops  contained  in  the 
Political  Power  Attrition  Sector. 

Information  Multipliers 

Imbedded  in  the  model  are  Information  Multipliers 
that  are  displayed  as  auxiliary  variables  at  points  of 
influence.  These  represent  places  where 
information  operations  can  be  used  to  magnify  the 
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effect  of  conventional  actions.  Learning  to 
capitalize  on  these  points  of  leverage  will  propel  the 
system  approach  of  power  into  the  future  giving  the 
Alliance  a  whole  new  dimension  from  which  to 
approach  security  issues. 

Model  of  Alliance  Power 

A  model  of  the  system  of  national  power  is  shown 
on  the  following  page  (Fig  7).  It  is  displayed  in  a 
visual  format  -  a  causal  diagram  which  allows  the 
decision  maker  to  see  the  interactions  between  and 
within  the  various  components  of  an  adversary’s 
power  base.  Key  points  of  leverage  have  been 
identified  as  positive  feedback  loops.  These  are  not 
all  inclusive,  but  they  are  the  pressure  points  that 
can  be  used  to  resolve  crisis  at  any  point  from 
identification  up  through  force  application. 

A  Model  as  a  Tool  for  Leadership 

The  system  level  model  illustrates  the  linking 
mechanisms  between  the  previously  described 
sectors.  Understanding  the  system  aspect  requires 
visualization  of  the  interactions  between  the  various 
sectors  of  the  model.  Although  a  crude 
representation  of  the  whole  system,  the  model  has 
immense  value  in  demonstrating  the  system  aspects 
of  national  and  Alliance  power.  It  serves  as  a  tool 
to  build  a  broad  set  of  reference  patterns  that  future 
leaders  may  use  in  complex  decision  arenas  - 
visualization  and  patterns  enable  recognitive  or 
naturalistic  decision  processes  (Czerwinski,  1998). 
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Figure  6:  Positive  Feedback  Loops  in  the  Political  Power  Sector 


Info-Kinetic  National  Power  Model 
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Figure  7:  Causal  Diagram  of  a  System  of  National  Power 


Military  Attrition  Sector 
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At  this  point,  we  can  update  the  chart  proposed  by 
Johnson  and  Libicki,  revealing  that  we  may  be  able 
to  use  visualization  models  as  aids  to  learning  — 
enabling  excursions  into  efficiency  levels 
previously  possible  only  through  dominant 
battlespace  knowledge  (Fig  8)  (Czerwinski,  1998). 

Developing  the  Model 

The  first  step  in  preparing  the  model  is  to  employ  a 
panel  of  experts  from  across  the  participating 
departments  and  agencies  to  more  fully  develop  the 
definition  of  the  model’s  variables  and  to  validate 
the  paired  relationships  between  them.  Once  this 
step  is  complete,  the  model  can  become  useful  in 
the  qualitative  analysis  of  policy  and  strategy 
options. 

The  second  step  is  to  develop  the  mathematical 
relationships  between  the  variables  and  to  use  the 
model  to  examine  historical  situations  to  calibrate 
the  model  for  specific  potential  adversaries.  This 
will  make  the  model  useful  in  quantitative  terms. 
Once  this  step  is  complete,  the  model  can  be  further 
incorporated  into  strategy  experiments  and  gaming. 

The  third  step  is  to  use  it  as  an  evaluation  tool  in 
programming  functions  to  test  alternative 
procurement  and  application  strategies.  The  results 
of  various  trials  could  be  used  to  “what  if’ 
scenarios  in  order  to  provide  leaders  with  an 
envisioned  pathway  towards  a  desired  state. 


Conclusion 

The  emerging  complexity  of  the  international 
security  environment  requires  a  broader,  more 
integrated  approach  than  was  required  in  the  Cold 
War  era.  This  challenge  can  be  met  by  an  integrated 
application  of  the  elements  of  national  and  Alliance 
power  enhanced  by  an  information  technology 
network  that  enables  collaborative  formulation  and 
execution  of  policies.  This  networked  approach 
will  create  a  more  complex  environment  for  leaders 
in  each  of  the  engagement  elements.  Decisions 
made  in  one  arena  will  have  direct  intuitive  and 
counterintuitive  downstream  effects  that  must  be 
understood.  Systems  Dynamics  offers  a  simple 
technique  that  can  be  used  to  map  the  system  of 
national  security  measures  enabling  an 
understanding  of  the  leveraging  effects  of 
networked,  information  based  warfare.  In  the 
future,  information  will  be  as  essential  to  warfare  as 
any  physical  weapon  system  technology.  With  a 
valid  model  to  serve  as  the  propelling  center  of 
thought,  information-based  warfare  will  develop  at 
rates  that  will  provide  a  renewable  competitive  edge 
for  the  Alliance. 


Potential  Value  of  DBK 


Figure  8:  Leveraging  Command  and  Control  Through  Aids  to  Learning 
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“Les  choses  simples  sont  difficiles  a 
expliquer”  Henri  Matisse. 

"If  you  know  the  enemy  and  know  yourself 
you  need  not  fear  a  thousand  battles.  If  you 
know  yourself  and  not  the  enemy,  for  every 
victory  you  will  suffer  a  defeat.  But  if  you 
know  neither  yourself  nor  the  enemy,  then 
you  are  a  fool  and  will  meet  defeat  in  every 
battle.  ”  Sun  Tzu  Circa  510  B.C. 

Abstract 

Intelligence  systems  are  designed  to  enable  self- 
knowledge  and  knowledge  of  the  opposition  to 
achieve  knowledge  superiority.  The  basis  for 
knowledge  is  the  collection,  collation, 
interpretation  and  dissemination  of  information. 
Superior  performance  in  the  marshalling  of 
information  sources,  the  creation  of  shared 
knowledge  and  the  projection  of  information  in 
future  plans  represents  what  has  been  called 
information  superiority  (Endlsey  and  Jones, 
1997). 

Almost  all  stages  of  knowledge  development 
involve  interpretation  and  the  introduction  of 
selective  filtering  or  emphasis.  The  integration  of 
computers  and  communication  systems  afford 


the  capability  to  share  large  volumes  of  highly 
processed  information.  There  is  a  significant 
difference  between  information  and  knowledge 
in  that  the  creation  of  knowledge  involves  the 
imposition  of  this  interpretive  framework  to 
organize  and  group  information  in  meaningful 
ways.  Often  information  must  be  filtered  to 
remove  noise  and  to  exaggerate  the  salient 
points.  In  addition,  assumptions  concerning  the 
validity  of  the  information  sources  must  be 
exercised  to  ensure  that  deception  and  sensor 
capabilities  are  taken  into  account. 

The  construction  of  this  shared  reality  amongst 
the  users  of  the  system  depends  on  social, 
cognitive  and  experiential  processes  guiding  the 
flow  and  management  of  information.  The 
situational  awareness  of  the  team  operating  the 
information  systems  is  a  vital  part  of  the 
effective  operation  of  the  system  because  it  helps 
to  direct  cognitive  resources  and  support  filtering 
of  task-and  context-irrelevant  information.  It  is 
possible  that  “the  knowledge  of  results  when 
attention  is  allocated  to  a  zone  of  interest” 
(Fracker,  1988)  are  poorly  articulated  across  the 
team  and  this  results  in  obfuscation  of  the 
prioritized  goals  or  the  context.  This  is  important 
because  of  the  likely  relationship  between 
situational  awareness  and  the  quality  of  the 
decision  making  process  (Endsley  and  Jones, 


1  This  report  has  been  written  wholly  by  the  first  author  and  he  accepts  full  responsibility  for  the  contents 
of  the  document.  However,  significant  elements  of  the  reports  were  influenced  by  numerous  discussions 
with  the  other  contributors  and  by  papers  submitted  by  co-authors  elsewhere. 


Paper  presented  at  the  RTO  HFM  Symposium  on  “Usability  of  Information  in  Battle  Management 
Operations  ”,  held  in  Oslo,  Norway,  10-13  April  2000 ,  and  published  in  RTO  MP-57. 
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1997).  Team  situation  awareness  is  more 
complex  than  additive  models  of  individual 
situational  awareness  as  some  have  suggested 
(Klein,  1993)  and  simply  aggregation  of 
collective  inputs  may  not  adequately  predict 
performance  outcomes  (Salas,  Prince,  Baker, 
Shrestha,  1995). 

The  design  of  any  large-scale  socio-technical 
system  for  sharing,  processing  and  managing 
information  resources  requires  careful  analysis  in 
the  development,  implementation  and  operation 
to  ensure  that  the  system  contributes  effectively 
to  the  performance  it  is  intended  to  support. 
Many  analyses  of  such  systems  focus  on  process 
variables  without  establishing  their  relationship 
to  outcomes.  For  example,  it  has  been  found  that 
communication  measures  may  actually  fail  to 
predict  performance  outcomes  because  they  do 
not  address  the  mediating  process  of  knowledge 
realization  across  the  team  (Cook,  Angus, 
Brearley  and  Stewart,  1998).  This  finding  has 
been  recently  supported  by  work  demonstrating 
that  communication  may  not  change 
significantly  even  though  the  cognitive  capacity 
to  use  the  information  exchanged  does  across  the 
day  (Reid,  2000).  In  addition,  it  has  been  shown 
that  more  commanders  may  actually  send  fewer 
messages  to  their  teams  but  spend  more  time 
planning  (Artman,  in  press). 

What  seems  to  be  critical  is  the  capacity  to 
process  information  further  to  create  the 
comprehension  stressed  by  Endsley  (1995,  1996) 
as  a  form  of  second  level  situation  awareness. 
This  is  in  accord  with  the  view  that  the  process 
of  making  or  creating  knowledge  for  decision 
making  is  more  instrumental  in  preventing  faulty 
decision  making  than  processes  of  review  given 
the  time  constraints  of  many  military  tasks 
(Cook,  Angus,  Brearley  and  Drummond,  1998). 
There  are  many  examples  in  the  literature  where 
the  processes  of  intelligence  development,  or 
operation,  or  both  have  been  poorly  managed 
with  catastrophic  consequences. 

It  can  be  argued  that  the  reasons  for  such  system 
level  failures  in  socio-technical  information 
systems  can  be  traced  to  three  perspectives  that 
are  inappropriately  expressed  in  the  design  and 
operation  (see  Flowers,  1996;  also  Luff,  Heath 
and  Greatbach,  1994).  The  first  perspective 
concerns  the  user(s).  As  Artman  (1999a)  has 
expressed  very  cogently  the  user  is  often 


denigrated  in  the  development  process  as  the 
root  of  many  problems  and  the  newly  developed 
system(s)  are  intended  to  manage  the  user’s 
behaviour  to  achieve  greater  levels  of 
performance.  Thus,  so-called  user-oriented 
design  is  only  a  rhetorical  statement  concerning 
the  existence  of  contact  with  the  user-population 
and  not  a  guarantee  of  the  effective  elicitation  of 
user  requirements  nor  is  it  a  tacit 
acknowledgement  of  the  central  role  of  users  in 
decision-making  processes. 

The  second  perspective  concerns  the  potential 
different  user  groups,  or  system  clients,  and  the 
way  that  information  sharing  is  prioritized 
among  the  different  groups  in  relation  to  their  air 
warfare  roles.  Air  warfare  in  the  form  of 
composite  air  operations  is  a  complex  process  of 
integrating  diverse  assets  effectively  and  one  of 
the  factors  that  may  determine  the  success  of  this 
annealing  of  assets  across  and  within  packages  is 
the  use  of  information  systems  (Directorate  of 
Air  Staff,  1999). 

The  third  perspective  concerns  the  effective 
operation  of  the  battlefield  information  systems 
in  a  diverse  range  of  contexts  both  in  space-time 
and  in  composition.  Failure  to  acknowledge  the 
large  array  of  factors  that  shape  performance  in 
the  use  of  information  systems  can  undermine 
the  operation  effectiveness  of  deployed  systems. 
Indeed,  it  can  be  argued  that  operational  issues 
and  limited  foresight  in  design  can  be  good 
examples  of  the  latent  pathogens  that  may 
generate  total  system  failure  in  the  complex 
systems  (Reason,  1990,  1997). 

Cunning  Cavemen  and  Dumb  Machines 

One  of  the  concerns  of  Artman  (1999a)  was  to 
draw  attention  to  the  assumptions  underpinning 
many  system  developers  views  in  relation  to  the 
user  population  their  systems  were  designed  to 
support.  Artman  felt  that  on  the  one  hand  there 
was  a  tendency  to  focus  on  the  negative 
attributes  of  the  human  and  on  the  other,  to  dwell 
upon  the  strengths  of  machine  intelligence.  This 
bias  in  the  reporting  of  capability  was  clearly 
problematic  for  a  number  of  reasons.  First,  the 
human  being  was  the  final  line  of  control  and  it 
was  the  human  being  that  would  take  the  crucial 
decisions.  Thus,  it  was  important  to  address  the 
user’s  needs  in  more  than  just  the  palliative 
sense,  which  many  information  technology 
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projects  do  at  present.  Second,  the  capability  that 
machines  have  is  more  focussed  on  basic 
information  processing  and  any  notions  of  so- 
called  intelligent  or  adaptive  function  are  both 
rudimentary  and  costly  to  validate  at  present. 
Thus,  the  thinking  and  deciding  is  left  to  the 
human  operators  and  managers. 

In  relation  to  total  system  function  there  is  no 
doubt  that  the  raw  and  unflagging  power  of 
computers  in  processing  information  is  valuable. 
The  new  links  between  computers,  that  enable 
high-speed  digital  exchange  of  information  via 
secure  networks,  are  both  attractive  and  valuable 
in  maximizing  the  cognitive  resource  utilization 
of  all  participants.  It  is  clear  that  “Operation 
Allied  Force  highlighted  the  blending  of  tactical 
implications  with  strategic  issues  -  the  blend 
being  in  the  cockpit,  where  the  pilot  is  the  final 
part  of  the  decision  loop.”  (Penney,  1999,  p.32). 
This  integration  of  the  front-line  crews  in  the 
decision  making  process  should  be  made  with 
care  to  ensure  that  the  limited  cognitive  capacity 
of  the  busy  pilot  is  not  overloaded  and  part  of 
that  process  must  include  a  credible  sharing  of 
information. 

Pilots  also  need  to  develop  confidence  in  their 
decision  making  because  they  need  to  feel  able 
to  take  decisions  without  balancing  constraints. 
Thus,  one  would  imagine  benefits  of  effective 
and  knowledgeable  support  from  fighter 
controllers  with  the  role-specific  knowledge, 
directing  the  information  flow  and  negotiating 
appropriate  actions  with  front-line  crews.  The 
concern  with  the  quality  of  the  fighter  controllers 
is  something  which  has  been  raised  before 
(McManners,  1 996)  and  it  shows  the  way  in 
which  the  cognitive  and  social  skills  of  the 
individuals  in  any  system  shape  the  performance 
outcomes.  It  is  clear  that  the  human  element  is 
important  because  of  the  way  that  human 
operators  are  often  called  to  account  for  failure. 
However,  human  and  system  operations  are 
intertwined  and  ineffective  design  can  generate 
inappropriate  performance. 

The  information  must  be  presented  to  human 
operators  in  a  form  and  at  a  time  in  which  it  is 
possible  and  feasible  to  influence  the  on-going 
course  of  events.  Human  operators  need  to  keep 
ahead  of  the  curve  and  when  they  fall  in  line 
with  it  or  behind  it  the  potential  for  erroneous 
decision  making  is  great.  At  present  the  use  of 


technology  fails  to  maintain  an  intelligently 
managed  dialogue  between  the  human  and 
machine  intelligence,  with  much  of  the  workload 
associated  with  dialogue  management  reliant  on 
the  somewhat  limited  capacity  memory  of  the 
human  operator.  This  is  even  true  in  second 
order  ways  because  operators  are  rarely  able  to 
interrogate  systems  for  histories  or  to  set  timers 
to  help  them  schedule  activities  at  some  future 
point  in  time.  The  passivity  of  the  machine  is 
deceptive  in  that  for  many  activities  the  pace  and 
temporal  aspects  are  human  driven.  It  is  possible 
that  the  lack  of  context  sensitivity  on  the  part  of 
machine  intelligence  is  a  major  element  of  the 
dialogue  between  human  and  machine.  Humans 
frequently  despair  of  the  inappropriateness  of 
machine  interruptions  and  the  banality  of  the 
requests  made. 

The  difficulty  in  designing  for  multiple  users 
with  differing  requirements  and  the  costs 
associated  with  disseminating  information  is  in 
provision  of  equipment,  equipment  support  and 
training.  This  operational  complexity  means  that 
there  is  a  tendency  to  centralize  information 
system  development,  focussing  on  the  needs  of 
the  key  decision-maker  or  leader  (Artman, 

1999).  This  process  of  centralization  may  lie  at 
the  heart  of  the  process  of  ineffective  decision 
making  in  fostering  the  conditions  for  high 
workload  and  lowered  situational  awareness.  In 
addition,  this  may  increase  the  risk  of 
Groupthink  because  it  may  tend  to  stifle 
participation  in  negotiated  decision  making. 

Even  in  military  systems  the  social  and 
contextual  world  is  a  constructed  experience 
which  is  negotiated:- 

“  A  team  of  agents  have  a  joint 
persistent  goal  relative  to  q  to  achieve 
(a 

belief  from  which,  intuitively,  the  goal 
originates)..  .In  short,  the  notion  of 
togetherness,  of  group  and  teamwork  is 
based  upon  the  notion  of  joint  persistent 
goals,  which  are  but  individual  goals 
associated  with  social,  namely  mutual, 
beliefs.”  Conte  and  Castelfranchi  (1995, 
p.  153). 

This  general  approach  to  command  and  control 
has  been  supported  by  a  recent  paper  from 
Artman  (2000)  in  which  negotiation  is  one  of  the 
mechanisms  used  to  share  information.  The  two 
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other  mechanisms  being  attentive  monitoring 
and  the  use  of  artefacts.  Those  operators  at  the 
lower  levels  of  command,  without  direct  access 
to  information  may  feel  dis-empowered  and 
unable  to  contribute  to  the  discussion  without 
access  to  the  relevant  information. 

The  dialogue  between  humans  and  humans,  and 
humans  and  machine  intelligence  or  agents  is  a 
clearly  a  crucial  part  of  the  operational 
effectiveness  in  that  human  cognition  is  situated 
in  an  operational  environment  or  domain  which 
stimulates  mental  activity.  The  more  passive  the 
user  becomes  the  less  effective  they  may  become 
in  relation  the  decision  making  in  the  assigned 
task  as  their  awareness  of  relevant  information 
collapses.  In  many  respects  this  acknowledges 
the  general  observations  made  concerning  the 
processes  involved  in  building  situational 
awareness  (Endsley,  1988)  and  the  importance 
of  active  information  capture. 

It  is  possible  that  inexpertly  designed 
information  systems  may  at  one  and  the  same 
time  encourage  decreasing  numbers  of 
exchanges  between  crews  and  afford  limited 
access  to  the  information  required  for  building 
knowledge.  Or,  the  number  and  type  of 
exchanges  may  vary  little  from  that  deployed  in 
previous  systems  but  the  ability  to  use  and 
comprehend  information  is  significantly 
impaired.  For  example,  the  workload  of 
managing  the  dialogue  may  increase  and  reduce 
the  amount  of  resource  available  for  further 
processing.  Or,  the  detailed  information  in  the 
exchanges  may  not  activate  the  appropriate 
elements  of  the  participants’  mental  models  and 
create  comprehension. 

Situational  Awareness  in  Information  Systems 

Situational  awareness  is  “perception  of  the 
elements  in  the  environment  within  a  volume  of 
time  and  space,  the  comprehension  of  their 
meaning,  and  the  projection  of  their  status  in  the 
near  future”.  The  time  horizon  for  these  events 
in  air  warfare  is  rapidly  changing,  as  there  is  a 
steady  shift  away  from  close  in  weapons.  The 
modem  air-to-air  capability  of  air-to-air  (Clancy, 
1995;  Francillon,  2000;  Ripley,  1994;  Spick, 
1995,  2000;  Thomborough,  1995;  Nordeen, 
1999)  and  air-to-ground  systems  (Clancy,  1995; 
Ripley,  1994,  1999;  Spick,  1995,  2000; 
Thomborough,  1995)  which  makes  it  is  an 


increasingly  demanding  environment  in  terms  of 
the  forecasting  of  future  events  and 
interpretation  of  the  current  context. 

The  recognition  of  the  interacting  elements  of 
the  design  equation  is  crucial  to  the  effective 
development  of  battlefield  information  systems. 
While  the  visible  coupling  of  the  current  context 
and  future  events  is  further  apart  in  imagined 
space  and  time;  the  actual  time  available  for 
effective  corrective  action  is  growing  shorter.  It 
may  also  weaken  the  ability  to  link  the  current 
actions  with  the  future  outcomes  and  generate 
more  uncertainty.  This  increasing  closure  of  the 
intelligence  cycle  in  information  systems  in 
terms  of  time  is  well  illustrated  by  the  use  of  the 
JSTARS  system  or  UAVs  in  the  Gulf  War  to 
directly  guide  the  actions  of  forces.  An  increase 
in  pace  and  ferocity  that  has  continued  with  the 
recent  campaigns  (Kromhout,  1999). 

“Thus,  the  J-STARS  aircraft,  which 
took  radar  images  of  enemy  troops 
moving  across  the  desert  from  a  safe 
distance,  were  able  to  patch  their  radar 
pictures  directly  to  the  operational  fire- 
control  cells  in  the  coalition  divisions 
below,  enabling  the  gunners  to  select 
and  engage  targets  at  will,  without  any 
reference  to  intelligence.”  Hughes- 
Wilson  Col.  J.  (1999)  Military 
Intelligence  Blunders,  p.349.  London  : 
Robinson  Books. 

This  quotation  may  illustrate  a  growing  tendency 
towards  an  increasingly  shorter  intelligence 
lifecycle,  short-circuiting  certain  elements  of  the 
process.  This  may  be  part  of  an  imperative  in 
practical  terms  to  attack  mobile  targets  that  can 
be  quickly  re-positioned  or  to  use  intelligence 
when  it  is  most  effective.  These  considerations 
for  speed,  a  vital  element  of  military  operations, 
and  the  increasingly  political  need  to  exert 
control  over  the  battlefield  represent  antagonistic 
forces.  There  are,  however,  very  good  reasons 
for  making  “real-time  information  directly  into 
all  cockpits”  (Warwick,  1999).  The 
implementation  of  this  information  network 
needs  careful  consideration  to  prevent 
unforeseen  consequences  at  both  higher  and 
lower  levels. 
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What  makes  military  information  systems  vital  is 
the  way  in  which  they  can  shape  situation 
assessment.  The  highly  dynamic  characteristics 
of  the  modem  battlefield  require  recurrent 
assessment  of  the  situation  to  identify  problems 
that  are  minor  and  insignificant  in  isolation  but 
can  develop  into  a  major  threat  (Sarter  and 
Woods,  1991).  The  continuous  process  of 
assimilating  new  information  is  cognitively 
resource  intensive  and  effortful,  and  the 
information  system  must  support  the  process 
effectively  to  prevent  the  operator  falling  out  of 
the  loop  or  behind  the  figurative  curve  on 
situation  developments. 

Training  Development  and  Mental  Models 

Despite  the  difficulties  in  measuring  mental 
models  it  is  clear  that  they  play  a  vital  role  in 
guiding  behaviour  in  complex  socio-technical 
systems.  Mental  models  are  defined  as  “an 
organized  knowledge  structure  that  includes 
objects,  situations,  and  events,  and  the 
relationships  between  them”  (Cannon-Bowers, 
Salas,  and  Converse,  1993).  There  is  evidence 
that  training  can  help  develop  more  effective 
team  exchanges  and  influence  team  performance 
(Stout,  Salas,  and  Fowlkes,  1997a,  1997b) 

Mental  models  are  important  in  supporting  the 
third  element  of  Endsley’s  Situational 
Awareness,  projection  of  future  events. 

However,  mental  models  may  aid  the  perception 
and  comprehension  of  events  as  well.  Mental 
models  are  products  of  experience  which  may  or 
may  not  be  revised,  re-evaluated  or  developed 
(Rasker,  Post  and  Schragen,  in  press).  It  is  to  be 
expected  that  mental  models  acquire  some 
degree  of  momentum  and  as  they  develop  they 
become  more  difficult  to  revise  substantially.  In 
energetic  terms  this  may  explain  the  resistance  to 
change  a  working  hypothesis  or  some  more 
fundamental  schema,  which  has  been  operational 
for  some  time  (Gilhooly,  1983). 

Cook  (2000)  has  proposed  a  more  elaborate 
explanation  of  the  typical  resistance  to  change  in 
a  model  adapted  from  Rasmussen’s  (1983,  1986) 
model  of  skill  development.  Rasmussen’s  model 
assumes  that  as  a  skill  develops  individuals 
change  their  mode  or  method  of  processing 
information  from  knowledge-based,  through 
rule-based  and  on  to  skill-based  processing.  This 
model  is  widely  used  in  the  literature  to  explain 


the  development  of  skill  in  the  management  of 
complex  systems. 

Cook  (2000)  has  proposed  that  emotional  and 
cognitive  gradients  encourage  the  skilled 
operator  to  try  to  remain  in  skill-based  mode  of 
operations  even  though  this  represents  a  non- 
optimal  approach  to  information  processing.  It  is 
proposed  that  the  experienced  operator 
experience  anxiety  when  they  are  forced  to  adopt 
rule  or  knowledge-based  processing  which  they 
interpret  as  aversive.  In  terms  of  a  two-factor 
theory  of  conditioned  avoidance  operators  are 
likely  to  learn  that  maintaining  skill-based 
processing  reduces  anxiety  induced  by  feelings 
of  loss  of  control  when  they  shift  to  rule-  or 
knowledge-based  processing.  At  the  same  time 
the  operator  will  paradoxically  feel  safer 
operating  in  a  skill-based  mode  of  processing 
because  they  experience  less  demand  on  their 
available  cognitive  resources.  This  tendency 
towards  focussed  processing  among  experienced 
operators  has  been  proposed  as  a  potential 
contributing  factor  to  accident  development  and 
may  result  in  failure  to  manage  more  unusual 
system  failures  among  experienced  operators, 
relative  to  less  experienced  (c.f.  Huey  and 
Wickens,  1993).  The  tendency  towards  less 
effortful  processing  has  been  described  many 
years  ago  as  satsificing  (Newell,  1955)  but  never 
associated  with  a  model  of  skilled  information 
processing. 

It  is  likely  that  both  social  and  cognitive 
processes  are  adaptive  in  complex  social 
technical  systems  such  as  that  associated 
battlefield  information  management  (Cook,  in 
press).  Recognition  of  the  part  played  by  the 
team  and  not  simply  by  the  leader,  in  processing 
information  and  arriving  at  a  decision  has  been 
strongly  emphasized  by  Artman  (1999a).  Artman 
(1999a)  has  criticized  the  tendency  towards  the 
use  of  drills  in  using  military  command  and 
control  systems  because  of  the  potential  shift 
towards  automaticity  of  actions.  Automaticity 
can  either  enhance  the  adaptability  and 
flexibility,  or  it  can  increase  the  vulnerability  of 
the  system  depending  on  the  rigidity  of  the 
organizational  culture.  It  is  important  to 
distinguish  the  use  of  drills  and  training 
experience  in  exercises  when  operators’ 
responses  and  capability  can  be  stretched  or 
developed.  Thus,  the  decision  to  engage  multi¬ 
asset  Airborne  Command  and  Control  systems  in 
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exercises  is  a  very  positive  step  towards  effective 
projection  of  force  in  future  engagements. 

Function  of  Intelligence  Systems 

“not  just  collection  but  of  collation, 
interpretation,  and  dissemination...” 
Hughes- Wilson  Col.  J.  (1999)  Military 
Intelligence  Blunders,  p.5.  London  : 
Robinson  Books. 

Battlefield  information  systems  have  the 
potential  to  capture  and  transmit  large  amounts 
of  information.  The  function  of  such  systems  is 
quite  close  related  to  intelligence  functions  as 
outlined  in  Figure  1  showing  the  intelligence 
lifecycle.  What  is  not  immediately  obvious  is 
that  the  process  of  delivering  information,  as  in 
any  medium,  can  result  in  distortion  because 
decisions  may  be  made  about  what  to  transmit  to 
whom. 

“For  the  professional,  intelligence  is 
simply  defined  as  processed,  accurate 
information,  presented  in  sufficient  time 
to  enable  a  decision-maker  to  take 
whatever  action  is  required.  Hughes- 
Wilson  Col.  J.  (1999)  Military 
Intelligence  Blunders,  p.5.  London  : 
Robinson  Books. 

All  of  the  constituent  processes  can  be  easily 
distorted  by  time  pressure  and  by  the  application 
of  knowledge.  The  distortions  can  be  effective 
exaggerations  that  reveal  the  priorities  for  action 
or  they  can  be  misleading  or  ambiguous  in 
obfuscating  the  true  intent  of  the  opposition.  The 
important  issue  is  the  ability  to  separate 
command  intent  and  capability  in  intelligence 
analysis.  The  recognition  of  error  propagation 
because  of  inferences  and  information  selection 
at  earlier  levels  in  mediated  systems  is  not  new 
(Mantovani,  1996)  and  careful  analysis  of  the 
socio-cognitive  aspects  of  the  system 
requirement  need  careful  development  by 
engaging  in  dialogue  with  the  stakeholders  of  the 
current  and  future  systems.  To  some  extent  the 
recognition  of  the  importance  of  the  intelligence 
team  has  been  recognized  in  other  domains  such 
as  emergency  command  and  control  (Artman 
and  Waem,  1999)  where  the  more  detailed 
analysis  of  interactions  has  been  applied  to 
similar  processes. 


There  are  cases  where  communication  support 
systems,  that  form  a  large  part  of  the  intelligence 
network,  have  decreased  the  overall  levels  of 
cooperation  and  consensus  building, 
undermining  coordination  and  effective  resource 
management  (Wickens,  Gordon  and  Liu,  1998). 
It  is  perhaps  not  surprising  that  more  information 
can  increase  the  quality  of  decision  making  and 
yet  it  can  decrease  the  confidence  and 
satisfaction  of  the  group  members.  According  to 
theoretical  model  of  skilled  performance 
perception  proposed  in  this  paper  more 
information  could  induce  anxiety  concerning  the 
ability  to  encapsulate  and  process  relevant 
information,  producing  uncertainty.  Any 
individual  might  deal  with  a  limited  part  of  the 
information  array  and  as  a  consequence  they 
might  not  feel  the  final  decision  represents  their 
views  effectively.  This  uncertainty  can  be 
managed  by  organizational  or  argumentative 
means.  Thus,  extending  the  discussion  might 
help  resolve  differences  as  all  the  relevant  items 
of  information  are  examined  in  turn  and  the 
interpretations  and  actions  considered,  or 
someone  may  be  nominated  as  the  final  decision 
maker  who  can  arbitrate.  Arbitration  may  be  a 
necessary  process  in  time-limited  safety  or 
mission  critical  decision-making  in  dynamic 
environment. 

Often  practitioners  forget  that  battlefield 
information  systems  simply  reveal  a 
representation  of  the  force  disposition  and 
capability  of  the  enemy.  This  image  of  the 
current  situation  is  complicated  by  the  use  of 
enemy  tactics  such  as  the  use  of  jamming 
(Dawes,  1999),  decoys  (Spick,  1999,  2000),  dis¬ 
information,  and  novel  strategies. 

“intelligence  officers  ....  who  were 
deceived  by  the  evidence  they  had  so 
conscientiously  collected  and  collated, 
and  who  failed  to  interpret  it  correctly- 
the  misinterpreters  ”  Hughes- 
Wilson  Col.  J.  (1999)  Military 
Intelligence  Blunders,  p.15.  London  : 
Robinson  Books. 


Both  the  accuracy  and  spatio-temporal  resolution 
of  the  imagery  is  limited  by  the  capabilities  of 
sensors  and  by  the  transmission  bandwidth 
available.  Thus,  the  operator  in  receipt  of 
information  must  qualify  the  image  by  the 
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application  of  knowledge  to  resolve  the  most 
likely  interpretation  of  the  image.  Then  in  the 
same  or  in  a  subsequent  process  they  must  infer 
the  enemy  intent  to  establish  a  course  of  action. 

If  the  operator  falls  behind  the  curve  in 
management  of  their  own  force  disposition  or 
application  of  their  power  then  the  gap  between 
success  and  failure  may  narrow  substantially. 

Intelligence  Systems  in  Team  Contexts 

It  has  been  noted  that  same  teams  operate  in 
variety  of  situations  and  coordination  of  these 
teams  is  critically  important  (Grimes,  2000). 
There  are  clearly  a  number  of  factors  that  can  be 
assessed  to  help  ensure  the  optimal  method  for 
sharing  knowledge  and  information  across  the 
team. 

First,  if  the  timing  of  information  sharing  is 
critical  this  may  determine  the  way  in  which 
information  flow  is  managed  and  who  receives 
what.  Where  time  is  critical,  as  in  air  warfare  and 
individual  operators  are  highly  skilled  it  may  be 
better  to  disseminate  raw  information  more 
widely.  However,  the  clients  receiving  the 
information  should  have  the  tools  and  the 
display  formats  appropriate  to  the  task  they  are 
required  to  accomplish  as  part  of  the  a  composite 
package  (Stapleton,  1999). 

If  timely  receipt  of  information  is  critical  then 
processing  bottlenecks  should  be  identified  in  all 
resources,  men,  cognitive  capability  and 
communication  linkages.  It  is  all  too  easy  to 
consider  the  process  of  dispatching  information 
as  an  automatic  process  and  to  fail  to  appreciate 
the  skill  involved  in  selecting,  packaging  and 
forwarding  information.  Only  recently  it  has 
been  publicly  announced  that  JSTARS  and 
AW  ACS  (Anonymous,  2000).  This  may 
represent  a  growing  awareness  that  AWACS  and 
ASTOR,  or  the  U.S.  equivalent  AWACS  and 
JSTARS,  in  combination  are  an  effective  force- 
multiplier  when  crews  are  trained  to  use  the 
systems  in  concert  effectively  (Anonymous, 
1999). 

Need  to  Know 

A  major  issue  in  intelligence  information 
collected  from  the  battlefield  concerns  who 
needs  to  know  and  what  they  need  to  know.  The 
reasons  are  numerous. 


Knowledge  of  what  one  knows  can  be  used 
inferentially  to  inject  decoys  and  mis¬ 
information  into  the  system  to  divert  resources. 
The  knowledge  concerning  what  is  known  may 
indicate  the  sensor  and  communication 
capability  of  the  total  information  system.  By 
knowing  the  accuracy  of  the  current  image  and 
its  weaknesses  it  may  be  possible  to  conduct 
psychological  and  information  war  in  which  the 
deployment  of  resources  in  strength  produces 
shock  or  information  overloads. 

For  whatever  reason,  the  use  of  intelligence 
information  is  usually  restricted  to  specific 
groups.  However,  this  focus  on  security  may 
lead  to  under-utilization  of  the  greatest 
battlefield  asset,  the  distributed  and  situationally 
aware  cognition  of  individual  operators.  No 
where  is  this  more  apparent  than  in  the  air 
warfare  environment  where  small  changes  in 
position  may  change  the  level  of  cognitive 
demand  on  operators. 

Imagine  a  large  package  moving  across  the 
Forward  Line  of  Troops  (FLOT)  when  the 
leading  aircraft  are  challenged  by  the 
deployment  of  high  capable  fighter  aircraft 
travelling  fast  and  high.  The  lead  elements  of  the 
package  may  be  focussed  on  the  rapidly 
advancing  threat  and  modem  missile  systems 
bring  them  within  target  range  very  quickly.  The 
elements  bringing  up  the  rear  may  be  more 
aware  of  Surface-to-Air  assets  and  conscious  of 
the  possibility  of  package  elements  being  forced 
down  into  or  around  onto  unseen  SAM  batteries. 
If  all  elements  are  privy  to  complete  air  picture 
through  JTIDS  or  an  equivalent  system  the 
response  may  be  more  effective.  There  may  be 
subtle  cues  to  the  crews  that  what  is  unfolding  is 
a  SAM  trap  and  on  that  basis  and  briefing  they 
may  formulate  a  more  effective  plan. 

The  free  availability  of  the  air  picture  mediated 
by  information  distribution  on  a  secure  network 
for  all  the  operators  may  help  to  free  resources 
and  make  the  outcomes  less  subject  to  the 
possibility  of  communication  jamming.  The 
cognitive  load  associated  with  communication 
may  be  diminished  and  more  resources  may  be 
available  for  the  specialist  roles  of  the  elements 
of  the  package.  Thus,  hard  and  soft  kill  EW 
elements  may  provide  effective  cover  for  the 
package  and  the  air  and  ground  elements  may 
have  the  confidence  to  progress  or  egress. 
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Communication  is  clearly  vital  in  establishing 
good  situational  awareness  and  it  has  been 
suggested  that  free  information  exchange  may 
help  to  compensate  for  limitations  within  teams 
(Bolman,  1979;  Orasanu,  1990;  Schwartz,  1990; 
Wegner  and  Simon,  1990).  However, 
communication  does  not  in  itself  guarantee 
superior  performance  (Cook,  Campbell  and 
Angus,  1998;  Reid,  2000). 

The  key  threat  to  effective  dissemination  is  the 
cost  of  the  individual  systems  making  the 
information  available  on  a  secure  network  and 
the  justification  of  that  cost-cutting  measure  on 
the  basis  of  security  and  prioritization  of 
resources  to  where  it  is  most  required.  From  the 
evidence  of  recent  conflicts  it  would  seem  that 
ground,  and  not  the  air  threat,  represents  the 
most  potent  enemy  asset.  Both  in  Iraq  and  in 
engagements  in  the  Former  Republic  of 
Yugoslavia  it  seems  that  that  very  few 
opposition  forces  will  be  able  to  effectively 
mount  a  coordinated  air  and  ground  defence 
against  hostile  aircraft  for  a  variety  of  reasons. 
However,  previous  conflicts  may  not  be 
representative  of  future  operational  requirements 
as  the  air  warfare  strategies  evolve  with 
knowledge  drawn  from  previous  encounters. 

There  is  no  doubt  that  the  transfer  of  information 
from  lower  echelons  upwards  and  from  higher 
echelons  downwards  may  create  problems  and:- 

“It  is  impossible  not  to  feel  some 
sympathy  for  those  unfortunate 
commanders  who  just  didn’t  know, 
because  someone  with  the  information 
failed  to  pass  it  on.”  Hughes- 
Wilson  Col.  J.  (1999)  Military 
Intelligence  Blunders,  p.15.  London  : 
Robinson  Books. 

Misinterpretation  can  propagate  these  errors  of 
information  omission  when  higher  echelons  seek 
on  information  confirming  their  expectations  or 
when  lower  echelons  selectively  prioritize 
information  they  feel  is  most  pertinent  to  the 
current  working  hypothesis.  It  is  well  known  that 
once  formed  a  working  hypothesis  is  difficult  to 
change  and  it  even  occurs  among  those  trained  to 
refute  hypotheses  i.e.  scientists  (Gilhooly,  1982). 
The  potential  for  biases  and  the  use  of  heuristic 
short  cuts  in  processing  large  volumes  of 


information  are  well  accepted  (Huey  and 
Wickens,  1993).  It  is  possible  that  these  errors 
may  not  propagate  as  effectively  when  all 
individuals  have  access  to  the  raw  data. 

Capability  and  Intentions 

“Understanding  the  differences  between 
a  potential  enemy’s  capabilities  and  his 
intentions  is  crucial  to  understanding 
the  difficulties  facing  the  purveyor  of 
intelligence.”  Hughes-Wilson  Col.  J. 

(1999)  Military  Intelligence  Blunders. 
p.5.  London  :  Robinson  Books. 

In  warfare  it  is  easy  to  see  the  opposition  as  a 
monolithic  force  that  one  faces  and  interprets  in 
a  coherent  way.  In  the  past  the  strongly 
controlled  Ground-Controller  Intercept 
management  of  air  warfare  favoured  by  the 
Former  Soviet  Union  was  seen  as  a  major 
weakness.  However,  no  unitary  asset  on  a 
battlefield  can  be  treated  as  sharing  a  common 
goal  of  a  centralised  leadership  because  it  is 
possible  that  they  have  developed  alternative 
strategies.  Indeed,  the  priority  goal  of  destroying 
the  enemy  command  and  control  are  likely  to 
ensure  that  the  approach  taken  by  different 
enemy  units  is  likely  to  be  more  fragmented  and 
less  coordinated  or  planned.  It  might  be  argued 
that  the  lack  of  a  decisive  policy  in  Kosovo 
concerning  targets  created  opportunities  for  the 
opposing  forces  to  escape  destruction  and  to 
ensnare  coalition  forces  in  traps  (Ignatieff, 

2000). 

The  centrality  of  leadership,  in  many 
frameworks  of  military  analysis,  seems  to 
reinforce  the  tendency  towards  fragmentation, 
segmentation  and  compartmentalisation  that  may 
be  indirectly  supportive  a  social  loafing  (Brown, 
2000;  Hartley,  1999;  Latane,  Williams,  and 
Harkins,  1979)  a  phenomenon  known  to  be 
counter-productive  in  group  interactions. 

The  important  issue  in  considering  these 
outcomes  is  the  consideration  of  who  is  best 
qualified  to  interpret  the  enemy  actions,  the  crew 
at  the  front  or  the  fighter  controller  circling  some 
distance  behind  the  FLOT. 

Interpretive  Skill  and  Self-Monitoring 
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There  may  be  a  tendency  to  focus  on  certain 
cues  from  the  developing  intelligence  in 
information  systems  and  for  this  to  generate  less 
than  optimal  performance  as  a  consequence  of 
risky  decision  making  or  conservative  decision. 
This  tendency  has  been  noted  elsewhere  in  the 
social  psychological  literature  where  faulty 
decision  making  based  on  a  risky  shift,  over¬ 
optimism,  Groupthink  and  polarisation  (Brown, 
2000,  Hartley,  1997)  are  well  recognized. 

Sir  Richard  Johns  (Chief  of  the  Air  Staff)  has 
warned  against  the  possible  deluge  of 
information  and  the  slow  transfer  of  information 
to  where  it  is  needed  (Penney  and  Doke,  1999). 
Sir  Richard  has  stressed  that  “intelligence  must 
be  100%  reliable  so  that  it  can  be  passed  rapidly 
to  the  targeting  and  attack  systems”.  This  view 
follows  the  tenor  of  the  view  regarding  the  use 
of  JSTARS  for  guiding  air-to-ground  operations 
and  UAV  aircraft  for  Battle  Damage  Assessment 
(BDA)  in  the  Gulf  War.  Implicitly  it  suggests 
that  the  capture  process  should  reduce  the  need 
for  interpretive  action  or  information 
management,  that  might  slow  down  the  process 
of  converting  information  advantage  into  tactical 
advantage. 

The  dissemination  of  information  is  a  key  issue 
in  the  development  of  battlefield  information 
management  systems.  The  doctrine  and  policy 
adopted  may  significantly  affect  the  performance 
outcomes  and  the  resource  utilisation.  It  has 
generally  been  agreed  that  there  are  difficulties 
in  field  operations  in  distributing  information 
and  that  may  be  significant  in  co-ordinating  and 
articulating  the  different  elements  of  the  total 
force  package. 

“It  has  gotten  better,  but  we  still  can’t 
get  down  to  the  company  level  what 
they  need  to  do  the  job”.  The  coalition 
forces  succeeded  in  the  Gulf,  but  the 
contribution  of  intelligence  during  the 
battles  was  sometimes  far  from  the 
definition  of  accurate  information 
passed  in  a  timely  fashion  to  decision¬ 
makers  to  enable  them  to  make  correct 
decisions.”  Hughes-Wilson  Col.  J. 

(1999)  Military  Intelligence  Blunders. 
p.5.  London  :  Robinson  Books 

The  concerns  with  distribution  of  information 
were  echoed  in  discussions  with  Stapelton 


(1999)  who  suggested  that  all  the  elements  of 
force  package  would  benefit  from  a  knowledge 
of  force  disposition  if  it  were  presented  directly 
to  the  operator  in  the  appropriate  format.  It  was 
underlined  that  the  reliance  on  voice 
communications  for  some  members  of  the  total 
package  strength  was  potentially  damaging  in 
two  ways.  First,  the  groups  with  better 
situational  awareness  (SA)  had  to  work  harder  to 
promote  SA  in  the  out-group  and  this  added  to 
their  workload.  Second,  those  without  systems 
promoting  SA  were  less  effectively  coordinated 
and  articulated  in  the  package  encouraging 
independent  action  which  was  less  than  optimal 
in  terms  of  the  total  battlespace.  Thus,  the 
introduction  of  partial  deployment  would  be 
counter-productive  and  may  in  some 
circumstances  generate  greater  losses. 

The  move  to  deployment  of  Beyond  Visual 
Range  (BVR)  weaponry  by  coalition  and 
opposition  assets  made  information 
dissemination  a  priority  issue  (Kromhout,  1999) 

.  expression  of  command  intent  and  authority 
could  be  muted  by  the  ineffective  deployment  of 
weapons  systems  in  an  environment  where 
concerns  about  fratricide  and  colateral  damage 
were  high,  as  they  normally  are  in  peace¬ 
keeping,  NATO  and  UN  force  deployments.  The 
move  to  beyond  visual  range  weaponry  has  been 
accompanied  by  increasing  sophistication  of  the 
cockpit  environment  (Coombs,  1999;  Aviation 
Week  and  Space  Technology,  1 996)  and  the 
avionic  systems  supporting  the  pilot  (Gunston, 
1990;  Rendall,  1997).  At  the  same  time  the 
number  of  operators  involved  in  the  system  has 
shrunk  to  increase  the  workload  on  those 
remaining  and  this  makes  team-effectiveness  an 
even  more  important  element  of  the  force 
equivalent  equation.  Cooperative  and  articulated 
asset  use  enables  higher  performance  outcomes 
from  the  same  absolute  number  of  assets  and  it 
should  decrease  risk  of  losses. 

It  is  clear  that  no  matter  how  much  the  system  is 
optimised  the  operators,  in  Airborne  Command 
and  Control  (ACC)  systems  or  in  the  front-line 
need  training  to  intelligent  execute  the  actions 
implied  by  the  interpretation  of  the  information. 
There  is  no  doubt  that  the  whole  process  could 
be  automated  but  the  danger  would  be  friendly 
fire  mishaps,  colateral  damage  and  political 
disintegration  of  coalition  forces. 
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Sir  Richard  Johns’  views  concerning  the 
confidence  in  the  information  presented  are 
mirrored  by  concerns  from  Artman  (1999b)  who 
noted  that  operators  who  lack  confidence  in  their 
sensing  will  either  spend  time  cross-checking  the 
information  they  receive  or  they  will  fail  to  use  it 
effectively.  These  domain  specific  issues  are  a 
strong  indication  that  problems  concerning  trust, 
confidence  and  uncertainty  raised  elsewhere 
(Thimbleby  et  al.,  1994)  are  endemic  to 
information  systems. 

Cook  (1999a,  1999b)  has  proposed  that 
confidence,  trust  and  uncertainty  are 
psychological  properties  of  all  physical  systems 
in  military  usage  and  they  can  distort  decision 
making.  Given  the  model  presented  earlier 
concerning  the  strategic  allocation  of  cognitive 
resources  in  experienced  operators  and  their 
aversive  experience  of  uncertain  situations,  or 
knowledge-based  modes  of  information 
processing,  it  seems  possible  that  strategic 
management  of  workload  may  be  guided  by 
affective  cues  and  cognitive  resource  gradients. 

It  is  surprsing  that  in  complex  control  situations 
it  may  be  possible  for  operators  to  fail  to  notice 
significant  differences  in  the  expected  and  the 
experienced  conditions  if  the  appropriate  tools 
are  not  available  to  aid  the  comparison  (Wood, 
2000).  This  suggests  that  most  operators  would 
have  only  a  broad  brush  impression  of  the 
situation  and  this  is  confirmed  by  the  analysis  of 
situational  awareness  in  complex  command  and 
control  situations  (Grimes,  2000). 

Conclusions 

The  use  of  technology  to  dominate  the  battlefield 
is  a  large  part  of  the  most  recent  conflicts  :- 

“The  second  element  was  provided  by 
computers.  When  linked  up  to  surveillance 
satellites  as  well  as  spy  planes,  computers 
increase  the  information  to  the  commander 
and  if  -  a  big  if  -  this  information  can  be 
digested  and  compressed  into  timely 
knowledge  of  the  enemy’s  dispositions, 
computers  can  improve  a  commander’s 
capacity  to  react  in  real  time.”  Ignatieff 
(2000)  Virtual  War,  London  :  Chatto  and 
Windus,  p.  171. 
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SUMMARY 

An  assessment  of  a  fully  operational  command 
centre  performing  under  high  pressures  in 
information  load,  time  stress  and  cognitive 
complexity  has  shown  that  in  particularly,  four 
factors  play  a  crucial  role:  individual  information 
processing,  team  management,  communication  load 
and  the  distribution  of  tasks.  It  is  concluded  that 
individual  processes  and  team  processes  compete 
with  each  other.  If  individuals  are  getting  loaded 
then  first  those  tasks  will  be  dropped  that  are 
demanding  and  do  not  lead  towards  direct  feedback. 
Team  tasks  suffer  most  under  these  conditions. 

KEYWORDS 

Command  centre,  team  performance,  observational 
methods,  performance  evaluation,  workload 

1  INTRODUCTION 

Recently,  the  Royal  Netherlands  Navy  has  launched 
several  studies  for  the  analysis  and  design  of  the 
operational  effectiveness  of  command  crew.  These 
aim  at  understanding  better  the  critical 
requirements  of  command  centres.  An  important 
driver  of  these  studies  is  the  need  to  optimise  crew 
size  and  operational  performance.  In  this 
presentation  I  will  focus  on  the  assessment  of  the 
current  M-frigate  class  command  centre  that  was 
performed  in  1998  by  a  team  of  researchers  of  TNO 
Human  Factors. 

The  attention  towards  command  crew  effectiveness 
is  one  that  fits  a  gradual  development  of  human 
factors  studies  for  the  Dutch  Navy.  These  had  an 
early  focus  on  workplace  ergonomics,  and 
developed  gradually  into  interface  design  and 
common  visual  spaces,  into  assessment  of  the 
human-machine  complex  in  the  current  command 
arrangement  and  for  future  command  centres.  In 
parallel,  efforts  are  being  employed  to  structurally 


incorporate  human  factors  in  the  broadest  sense  into 
the  conceptual  development  and  the  design  process 
(Human  Systems  Integration,  US  DOD  5000.1, 
1992;  UK  Human  Factors  Integration  Program, 
D/DOR  (Sea),  1991;  Beevis,  D.,  Essens,  P.  & 
Schuffel,  H.,  1996). 

Command  centres 

A  command  centre  (operations  room,  or  combat 
information  centre)  of  a  frigate  is  a  typical  example 
of  a  complex  system  where  information  from 
different  ‘worlds’  (air,  surface  and  subsurface)  is 
gathered,  analysed  and  acted  upon. 

A  command  centre  can  be  characterised  as  a 
’human  activity  system’  (Checkland,  1993)  designed 
for  human  information  processing,  decision  making 
and  execution  supported  by  technologies  such  as 
interfaces  to  sensor  and  weapon  systems,  and 
combat  information  system.  The  system  is  built  to 
respond  to  a  wide  variety  of  situations  and  signals 
from  external  worlds,  that  are  dynamic  and  change 
with  varying  time  horizons.  The  system  is  complex 
because  it  is  comprised  of  multiple  people  and 
multiple  technologies  organised  in  several 
subsystems.  These  have  to  tune  their  processes  and 
combine  and  collate  the  information  produced  in 
order  to  achieve  the  mission  of  the  ship. 

Traditional  human  factors  evaluations  of  complex 
prototypes  or  operational  systems  usually  focus  on 
operator  performance  instead  of  performance  of  the 
whole  system  (Meister,  1998),  or  limits  evaluation 
to  a  sub-system,  such  as  the  human-computer 
interface,  or  to  a  single  criterion,  such  as  workload. 
Outcome  measures  of  the  whole  system  may  be 
used  to  get  an  summative  view  on  the  performance 
of  the  whole  system  (often  referred  to  as  MOEs). 
The  problem  is  however  that  a  systems  outcome 
approach  treats  the  system  as  ’black  box’  (input- 
output)  hiding  multiple  internal  sources  of  positive 
and  negative  effects  which  lead  to  some  composite 
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result.  Therefore  these  measures  fail  to  provide  an 
understanding  of  the  factors  that  generate  that 
outcome.  The  fundamental  problem  however  is  that 
there  is  still  insufficient  knowledge  of  the 
contribution  of  one  factor  to  the  outcome  of  the 
whole  system  or  what  factors  contribute  most. 

2  M-FRIGATE  COMMAND  CENTRE 

The  study  of  the  current  command  arrangements  in 
the  M-ffigate  -  the  focus  of  this  paper  -  was  driven 
by  the  Navy’s  need  to  understand  better  which 
factors  are  critical  for  effective  performance  in 
order  to  optimise  current  command  and  to 
incorporate  those  in  the  design  of  future  ship  and 
command  centres. 

A  command  centre  in  full  operation  usually 
displays  an  impressive  hectic  and  concentrated 
activity  often  with  intense  verbal  communications 
via  networks  and  face-to-face.  Assessment  of 
performance  under  these  conditions  requires  a  well 
developed  methodology  and  instruments  that 
capture  the  dynamics  of  the  processes  and 
addresses  the  critical  factors  of  the  complex  system. 
Our  tasks  was  to  develop  such  methodology. 

Of  other  studies  in  Naval  command,  in  particular 
the  TADMUS  studies  should  be  mentioned 
(Cannon-Bowers  &  Salas,  1998).  Similarly  to  goals 
in  our  command  studies  they  study  the  complex 
nature  of  command  in  dynamic  and  stressful 
situations.  A  difference  is  that  they  have  focused  on 
the  air  defence  team,  while  we  look  at  the 
command  centre  as  a  whole  representing  the 
mission  of  the  ship. 

Moreover,  the  M-ffigate  assessment  was  directed 
specifically  to  the  factors  that  determine 
effectiveness  of  the  current  command  centre.  This 
will  then  be  used  as  a  baseline  for  further  studies. 
The  assessment  does  not  address  the  strategic  or 
tactical  outcome  of  decision  making,  nor  the 
performance  of  the  technologies  used,  rather  it 
focuses  on  individual  and  team  processes  in 
information  processing  and  decision  making. 

Performance  indicators 

High  workload  conditions  are  considered  to  be 
‘normal’  in  an  operational  command  centre.  The 
question  is  whether  workload  levels  are  that  high 
that  processing  capacity,  of  individual  or  team, 
shows  its  limits.  This  would  negatively  affect 
performance  effectiveness  -  there  is  more  to  be 
done  than  can  be  handled  in  the  time  given.  A 
second  question  is  whether  workload  is  evenly 


distributed  over  the  crew.  If  in  a  critical  situation 
some  team  members  have  not  much  at  hand  while 
others  are  overloaded,  then  resources  are  apparently 
under-utilised.  But,  not  only  from  an  operational 
perspective  is  workload  an  important  issue.  Also 
for  the  crew’s  appreciation  and  quality  of  work: 
constant  performing  below  expectations  will  result 
in  negative  feelings  and  consequently  in  lower 
performance. 

In  dynamic  event-rich  situations  measuring 
workload  once  at  the  end  of  a  mission  would  be 
insufficient.  A  continuous  review  of  the  workload 
gives  insight  how  workload  develops  over  time  for 
each  crew  member,  where  bottlenecks  are  in 
processing  capacity,  and  how  workload  is 
distributed  over  the  crew  members. 

Workload  can  be  used  as  an  indicator  of  efficient 
and  effective  team  performance  but  does  not  show 
how  well  the  job  is  done.  Qualification  of  how  well 
a  task  is  being  performed  requires  insight  in  how 
the  demands  of  an  operational  situation  are 
addressed.  In  complex  tasks  one  should  also  take 
into  account  that  there  are  parallel  tasks  to  be 
processed  and  prioritisation  or  effective  task 
switching  can  only  be  judged  in  context.  This  is  the 
role  of  the  domain  expert.  Judgement  of  quality 
should  be  coupled  with  critical  events  in  the 
scenario’s  which  makes  it  easier  to  score  during  a 
sessions. 

METHOD 

Experimental  setting 

Obviously,  in  order  to  assess  the  effectiveness  of  a 
command  centre  under  critical  conditions  one  needs 
a  realistic  setting,  with  realistic  scenario’s,  with  a 
team  that  is  fully  operational. 

On  the  other  hand,  for  producing  well-founded 
conclusions,  one  needs  control  over  the  situation, 
repeated  measurement  using  multiple  teams 
responding  to  the  same  situations.  Moreover,  given 
time  and  resources  available  this  has  to  be  done  in 
an  efficient  way. 

We  have  used  the  Navy’s  training  facility  which 
has  a  copy  of  the  M-frigate  command  centre.  The 
use  of  matching  equipment  and  layout  facilitates 
the  application  of  skill-based  activities  so  that  the 
teams  can  perform  the  way  they  are  used  to;  only 
the  variations  in  the  scenario’s  will  show  in  the 
performance. 
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The  use  of  a  land-based  facilities  always  evokes 
discussions  of  the  effect  of  ship  movements  on  the 
performance.  Indeed,  ship  movements  result  in 
fatigue  which  affects  performance.  Moreover,  there 
may  be  effects  on  team  tasks  such  as 
communication  and  team  management.  A  second 
aspect  of  reality  not  easily  simulated  in  a  training 
facility  (and  in  standard  sea-based  exercises)  is  the 
stress  of  being  under  attack.  It  was  argued  that 
these  factors  would  certainly  add  to  performance 
degradation,  but  that  first  stress  due  to  time-limits 
and  information-uncertainty  should  be  investigated. 
In  order  to  augment  the  validity  of  the  assessment, 
the  findings  from  the  experimental  situations  were 
verified  during  full  sea  exercises. 

Registration  of  workload 

All  workstations  were  equipped  with  a  special 
keypad  with  five  keys  representing  a  5-point  scale: 
Most  left  key  represents  score  1  defined  as  ‘time 
left’;  most  right  key  defined  as  score  5  indicating 
‘too  little  time’.  The  middle  key  (3)  was  defined  as 
‘just  enough  time’.  The  operator’s  task  was  to 
indicate  how  much  time  was  left  for  doing  their 
jobs  at  that  very  moment.  Every  five  minutes  a 
small  lamp  on  the  keypad  flashed  which  cued  the 
team  member  to  give  a  judgement.  Each  key  press 
was  registered  in  a  computer-system  and  linked  to 
video  registration  (Observer  system). 

Registration  of  performance  quality 

Four  indicator  categories  are  distinguished:  the 
quality  of  the  information  processing  of  individual 
crew  members,  the  communications  between  the 
team  members,  the  handling  and  application  of  the 
systems  and  tools,  and  the  management  of  team  by 
the  team  leader.  The  categories  cover  several 
subcategories: 


‘Information  processing’  comprises  delay  in 
information  processing,  overlooking  of 
information,  errors  in  interpretation; 
‘Communications’  concerns  the  correct 
application  of  communications  procedures, 
timely  communications,  interrupting 
communications; 

‘System  handling’  covers  the  ease  of  operating 
a  system,  lack  of  skills  in  operation;  adequacy 
of  settings; 

‘Team  management’  refers  to  limitations, 
adequacy  or  overdoing  in  direction  and  control. 
Seven  domain-experts  were  distributed  over  the 
team  and  observed  performance.  When  they  judged 
during  the  session  that  one  or  multiple  categories 
were  applicable  in  that  situation  this  was  scored  on 
a  form  and  with  a  key  press  the  instance  and  its 
weight  was  linked  to  the  video  recording. 

Scenario’s 

Scenario’s  are  sequences  of  predefined  conditions 
and  events.  Assessment  of  a  system  requires  critical 
events  of  combination  of  events  that  load  of  even 
test  the  limits  of  the  central  functions  of  a  system. 
For  a  command  centre  central  functions  are 
processing  information  and  making  decisions. 
Three  scenario’s  were  developed:  a  normal 
workload  scenario’s  of  which  was  expected  that  a 
well-trained  team  could  handle  without  problems; 
an  input  information-load  scenario  with  many 
tracks  and  information  units;  and,  an  information- 
uncertainty  scenario  with  missing  information  and 
unexpected  behaviour  of  the  objects  in  the  world. 

Scenario  design  was  based  on  our  knowledge  of 
factors  that  affect  information  processing  and 
decision  making,  such  as: 

number  of  tracks  and  multiple  threats  or  targets, 
and  number  of  potential  measures  relate  to 
overload  of  information  handling; 
conflicting  information,  ambiguous  cues, 
threats,  multiple  options  and  criteria, 
conflicting  use  of  resources  relate  to  cognitive 
stress  and  processing  capacity; 
insufficient  information,  misleading  or  novel 
situations,  limited  feedback  relate  to 
availability  of  knowledge  and  experience. 

Procedure 

Four  experienced  and  fully-operational  teams  of  16 
members  each  participated  in  the  assessment 
sessions  performing  in  three  scenario’s.  At  the  start 
of  the  session  they  received  a  standard  operational 
briefing  which  explained  the  operational  situation 
and  mission.  Also  was  explained  that  measurement 
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tools  were  used  to  get  an  indication  of  the 
functioning  of  the  command  centre  and  that  no 
individual  scores  were  isolated.  The  scenario 
sequence  was  the  same  for  each  team:  normal, 
information-load,  information-uncertainty  scenario. 
Each  scenario  took  about  40  minutes.  Between  each 
scenario  was  a  break.  After  the  session  a 
questionnaire  was  handed  out  which  addressed  the 
functioning  of  the  command  centre  on  board. 
Subsequently,  in  a  debrief  scenario  events  and 
typical  performance,  actions  and  decisions  were 
discussed  by  the  team. 

RESULTS 

Workload  data  showed  that  about  half  of  the  scores 
are  around  the  middle  level  (‘just  time  enough’) 
which  can  be  considered  as  the  standard  level  of 
load  during  operations.  One  third  of  the  scores  was 
below  that  level  indicating  that  there  was  ample 
time  for  performing  the  tasks  at  hand.  One  sixth  of 
the  scores  are  above  the  just  enough  level 
indicating  that  there  were  instances  that  there  was 
very  little  time  to  perform  the  task  at  hand.  Overall 
levels  of  the  scores  can  be  clustered  around 
subgroups  in  the  command  centre.  It  was  found  that 
the  command  team  has  on  average  a  higher 
workload  then  the  supporting  operators,  with 
exception  of  a  particular  subgroup  of  operators  (the 
electronic  warfare  operator  and  controller)  who  had 
also  a  high  workload.  There  was  little  difference 
between  the  two  test  scenario’s.  The  conclusion  is 
that  the  unequal  distribution  of  workload  found 
results  from  an  organisation  of  tasks  which,  under 
complex  conditions,  centralises  control  with  the 
command  team. 


too  little 
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Quality  assessment  of  performance  showed  several 
opportunities  for  improvement.  Of  most  interest  is 
the  distribution  of  the  scores  over  the  categories. 
The  domain  experts  particularly  judged  that  team 
direction  and  control  (35%)  and  individual 


information  processing  (30%)  could  be  improved; 
less  prominent  were  system  handling  (18%)  and 
communications  (17%). 

The  questionnaire  showed  that  internal 
communications  was  considered  to  require  a  high 
level  of  attention  (90%  of  respondents,  N=53),  and 
that  its  volume  was  considered  to  be  too  high 
(60%).  The  tasks  to  be  performed  are  in  general 
time  critical  and  require  high  attention  (88%). 
System  handling  could  be  improved  by  improving 
the  interface  with  more  direct  interaction.  In  time 
critical  situations  any  system  delay  is  considered 
hindering  performance. 

DISCUSSION 

The  assessment  of  the  command  centre  performing 
under  high  pressures  in  information  load  ,  time 
stress  and  cognitive  complexity  shows  that  in 
particularly,  four  factors  play  a  crucial  role: 
individual  information  processing,  team 
management,  communication  load  and  the 
distribution  of  tasks.  An  interesting  statement  from 
a  Naval  evaluator  describes  the  combination  of 
these  factors  most  adequately:  ‘when  load  increases 
in  the  team  curtains  seem  to  get  shut  between  team 
members’. 

In  trying  to  understand  the  combination  of  factors 
we  distinguished  between  the  structural  settings  of 
the  command  centre  and  the  command  and  control 
process  itself.  Structural  is  the  given  quality  of  the 
team  and  their  ability  to  work  together.  Structural  is 
the  distribution  of  tasks  and  the  work  procedures 
that  are  predefined.  Structural  is  also  the 
technology  used  to  support  or  enable  the  command 
centre  to  do  its  job.  These  factors  are  brought  into 
play  before  the  mission  starts. 

During  the  operations  people  are  the  flexible  and 
vulnerable  force  and  adopt  compensation  strategies 
for  what  is  structurally  imperfect:  a  situation  most 
people  recognise  immediately.  From  the  literature 
on  stress  handling  a  checklist  of  strategies  was 
derived  (Gaillard,  1996): 

first  people  try  to  compensate  for  situations  the 
system  was  not  designed  for  by  putting  extra 
effort  into  play 

if  load  rises  further  tasks  for  which  that  is 
possible  are  delayed  or  postponed 
subsequently,  focus  is  on  a  limited  set  of  tasks; 
at  best  the  central  and  most  critical  tasks 
less  communications  is  observed  with  an 
increasing  tolerance  for  errors  (correction  costs 
time  and  effort) 
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finally,  control  over  other  people  is  loosened 
and  eventually  lost. 

This  list  shows  the  process  that  can  also  be 
observed  in  highly  loaded  conditions  in  command 
centres.  It  shows  that  individual  factors  and  team 
factors  interact  with  each  other  in  an 
understandable  way.  If  individuals  are  getting 
loaded  and  then  first  those  tasks  will  be  dropped 
that  are  demanding  and  do  not  lead  towards  direct 
success  or  hinder  their  most  direct  responsibilities. 
Team  tasks  can  be  considered  to  be  such  tasks.  In 
particular  when  team  leaders  have  also  other  tasks 
than  direction,  control  and  co-ordination  the  danger 
is  that  they  will  do  the  direct  accountable  tasks  first 
and  postpone  other  tasks  to  later.  Eventually  this 
may  result  in  a  breaking  down  of  the  team  as  a 
whole. 

In  conclusion,  the  following  factors  are  considered 
critical  for  effectiveness: 

-  Organisation  of  processing  capacity:  focussed  on 
optimised  task  distribution;  flexibility  in  task 
allocation;  reduced  communications 

-  Quality  of  people  /  team:  focussed  on  training  as 
a  team;  training  graceful  degradation  in  order  to 
deal  with  overload  situations;  training  in  complex 
and  varied  situations 

-  Technology:  focussed  on  directness  in  interaction 
and  response;  support  in  handling  information. 
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SUMMARY 

The  effectiveness  of  a  command  centre  largely  depends 
on  the  effectiveness  of  the  team  that  keeps  it  going.  This 
paper  describes  a  framework  of  five  research  methods  to 
investigate  command  centre  teams.  These  methods 
comprise  modelling,  observing,  experimentation,  design, 
and  evaluation.  Modelling  implies  breaking  up  a  whole 
human-human-machine  system  into  essential  elements.  It 
is  an  analysis  resulting  in  a  clear  description  of  the 
system.  Observation  is  needed  to  identify  possible 
bottlenecks  in  the  command  centre.  It  yields  insight  in 
the  composite  set  of  factors  that  influence  the 
effectiveness  of  the  command  centre  team.  Single  factors 
can  be  investigated  systematically  by  experimentation 
using  a  contrived  experimental  task.  The  knowledge  that 
is  gained  by  modelling,  observation  and  experimentation 
can  lead  to  the  design  of  a  new  command  centre  team,  or 
the  redesign  of  current  ones.  Finally,  any  particular 
design  may  need  an  evaluation  to  determine  how  team 
performance  is  effected.  The  application  of  the 
framework  is  illustrated  by  a  number  of  research 
projects. 

KEYWORDS 

Command  centre,  teams,  research  methods 

1  INTRODUCTION 

The  ability  of  teams  to  work  effectively  is  a  prerequisite 
in  a  number  of  critical  work  environments,  including 
military  command  centres,  fire-fighting,  aircraft 
cockpits,  emergency  medicine,  and  air  traffic  control. 
Such  teams  often  have  to  perform  in  complex  situations 
that  are  characterised  by  time  pressure,  heavy  workload, 
ambiguous  information  presentation  and  a  constantly 
changing  environment.  High  stakes  are  involved  and 
poor  performance  may  lead  to  considerable 
consequences. 

Many  studies  demonstrate  the  importance  of  teamwork. 
For  example,  in  the  aviation  domain  several  studies  have 
shown  that  many  incidents  and  accidents  are  due  to 
miscommunication  of  the  flight  crew  (Helmreich  & 
Foushee,  1993).  The  accident  with  the  USS  Vincennes 
has  been  attributed  to  ineffective  teamwork  (Klein, 
1993).  Heath  &  Luff  (1992)  show  that  effective  crisis 


management  in  the  London  underground  line  control 
room  depends  on  how  operators  exchange  and  monitor 
information.  Finally,  in  the  medical  world,  ineffective 
teamwork  has  led  to  a  considerable  number  of  incidents 
in  anaesthesia  (Howard,  Gaba,  Fish,  Yang,  &  Samquist, 
1992)  These  studies  show  the  importance  of 

investigating  teamwork  to  identify  the  factors  that  makes 
a  team  effective. 

The  focus  of  this  paper  is  the  research  on  command 
centre  teams.  A  command  centre  team  is  defmed  as  a  set 
of  at  least  two  people  that  work  together  toward  a 
common  goal,  who  have  been  assigned  to  specific  roles 
or  tasks  to  be  performed,  and  where  the  completion  of 
the  mission  requires  dependency  among  team  members 
(Dyer,  1984;  Salas,  Dickinson,  Converse  & 

Tannenbaum,  1992).  Central  in  our  approach  is  that  the 
command  centre  is  viewed  as  a  complex  cognitive 
system  in  the  sense  that  it  takes  situation  specific 
information,  knowledge  from  training  and  experience, 
mental  constructs  (hypotheses  and  assumptions),  and 
norms  and  values  that  are  combined  into  new 
information  entities  (Essens,  Post,  Rasker,  2000).  The 
core-business  of  a  team  in  the  command  centre  is 
information.  Signals,  datalinks,  intelligence  and  so  forth 
must  be  received,  interpreted,  and  assessed  in  order  to 
decide  upon  the  righteous  action.  Because  tasks, 
information  sources,  and  expertise  is  assigned  to 
different  team  members,  interdependent  interaction  is 
needed.  This  interdependency  requires  team  members  to 
engage  in  teamwork  behaviours  such  as  communication 
and  co-ordination.  The  challenge  is  to  determine  how 
effective  teamwork  can  be  realised  in  complex  systems 
such  as  the  command  centre. 

The  appreciation  that  teamwork  must  be  seriously 
studied  is  shown  by  the  variety  of  research  questions  that 
come  from  practice.  A  first  type  of  question  is  related  to 
the  complexity  of  a  command  centre  representing  a  need 
for  a  clear  description  and  understanding  of  this  human- 
human-machine  system.  Questions  such  as,  “which  tasks 
are  performed  and  which  knowledge  is  needed”  and 
“what  is  the  information  flow  in  the  command  centre  and 
how  is  this  exchanged  by  team  members”  represent  this 
need.  A  second  type  of  question  arises  out  of  problems 
that  are  experienced  in  practice.  Often  there  is  a  vague 
idea  that  something  is  wrong,  but  one  cannot  lay  hold  on 
the  specific  bottlenecks  and  the  seriousness  of 
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bottlenecks.  This  is  represented  by  questions  such  as 
“are  there  problems  with  the  workload?”  or  ”do  we  train 
our  personnel  well  enough?”.  A  third  type  of  question 
comes  from  a  need  to  develop  a  deeper  understanding  in 
the  underlying  factors.  Questions  such  as  “which  type  of 
knowledge  is  needed  to  communicate  efficiently”  or 
“what  is  the  effect  of  cross-training  on  co-ordination” 
represent  this  need.  A  forth  type  of  question  is  associated 
with  the  design  of  new  concepts,  systems  or  layouts. 
This  is  represented  by  questions  such  as  “how  must  the 
command  centre  be  equipped”  and  “what  is  the  best 
layout  of  the  command  centre  to  support  effective 
human-human  interaction”.  Finally,  the  last  type  of 
question  arises  from  a  need  to  evaluate  whether  the 
designed  concepts,  systems  and  layouts  are  applicable 
and  effective  in  the  real  world. 

The  objective  of  this  paper  is  to  describe  in  a  framework 
how  several  research  methods  can  be  applied  to 
investigate  command  centre  teams.  These  methods  are 
used  to  find  answers  for  the  various  types  of  questions 
stated  above.  With  these  methods,  teamwork  can  be 
investigated  in  command  centres  as  well  as  in  any  other 
complex  human-human-machine  system.  We  will 
illustrate  the  framework  with  a  number  of  research 
projects  and  research  techniques.  First  the  framework  as 
a  whole  is  explained,  and  then  each  method  is  discussed 
separately. 

2  RESEARCH  FRAMEWORK 

For  investigating  team  effectiveness  in  complex  systems 
we  describe  a  research  framework  consisting  of  five 
complementary,  but  individually  applicable,  methods 
that  span  all  phases  of  team  research  and  development. 
The  five  methods  are  modelling,  observation, 
experimentation,  design,  and  evaluation.  Figure  1 
presents  this  research  framework. 


Modelling  is  needed  for  gaining  understanding  of  the 
complex  environment  in  which  the  team  has  to  operate. 
Modelling  yields  a  description  of  the  essentials  of  a 
team,  such  as  its  structure ,  the  links  with  the  broader 
organisation  in  which  the  team  has  to  operate,  the 
functions  that  has  to  be  fulfilled,  the  tasks  that  the  team 
members  have  to  carry  out,  the  knowledge  they  need  for 
that,  and  the  types  of  information  that  flows  within  the 
team.  The  resulting  team  model  provides  the  set  of 
descriptors  for  all  team  entities  that  are  subject  of  further 
team  research  and  development.  Once  the  team  is  well 
enough  understood,  one  is  able  to  make  a  useful 
observation  of  an  operating  team.  Observation  deals  with 
the  actual  team  process,  revealing  a  problem  description 
in  terms  of  possible  bottlenecks  and  challenges  for 
improvements.  During  observations,  many  facets  play  a 
role  at  the  same  time,  and  linking  a  problem  to 
{interaction  of)  particular  factor  is  not  always  possible. 
With  experimentation,  where  a  team  environment  can  be 
controlled,  the  contribution  of  separate  factors  for  team 
effectiveness  can  be  identified.  New  initiatives,  or 
observed  bottlenecks  and  challenges  in  an  existing  team 
lead  to  team  (re)design.  Design  encompasses  the 
application  of  all  factors  of  team  effectiveness  (known 
from  own  research  as  well  as  others)  to  new  concepts 
and  detailed  realisations.  Finally,  a  designed  team  may 
be  evaluated  to  acquire  objective  team  performance 
measures  in  terms  of  speed ,  workload ,  quality ,  and 
quantity. 

Each  method  has  a  set  of  research  and  development 
techniques  associated  to  it.  The  next  sections  describe  de 
application  of  each  method  in  several  projects,  and  how 
they  are  supported  by  the  available  techniques. 

3  MODELLING 

The  functioning  of  a  command  centre  team  is  not  easy  to 
understand.  For  an  outsider  it  probably  seems  like  magic 
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Figure  1:  Research  framework  for  team  research  and  development 
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how  a  team  does  its  job,  using  an  overwhelming  amount  By  using  a  hierarchical  approach  in  modelling,  we  are 

of  displays,  push  buttons  and  communication  equipment,  able  to  finish  at  any  hierarchical  level  and  still  have  a 

communicate  with  strange  abbreviations,  being  quiet  at  complete  description.  If  necessary,  specific  end  nodes  of 
one  time,  and  completely  occupied  the  next  moment.  the  hierarchy  can  be  modelled  in  more  detail  when 

The  purpose  of  modelling  is  to  develop  an  understanding  bottlenecks  are  experienced  (or  established  by 

of  this  complex  team  and  its  environment.  This  is  done  observation)  in  that  part  of  the  hierarchy, 
by  reducing  its  complexity  through  an  analysis  and 
description  process.  By  analysis,  the  whole  system  is  Project 
broken  up  into  its  essentials.  This  is  a  complex  process 

itself,  because  the  essentials  and  their  relationships  are  This  method  is  applied  to  model  the  command  centre  of 

not  readily  given.  Often,  they  have  to  be  abstracted  from  the  Royal  Netherlands  Navy  (RNLN)  multipurpose 

the  many  details,  and  the  abstract  entities  that  arise  need  frigate.  Knowledge  about  the  task-related  structure  was 

to  be  labelled  and  described.  By  doing  so,  modelling  elicited  from  domain  experts  during  group  sessions.  This 

provides  a  clear  team  description  and  an  easily  forced  the  researchers  to  specify  precisely  their 

accessible  “map”  of  the  command  centre  team.  understanding  of  the  system.  The  large  quantity  of 

information  (changes,  annotations,  new  branches) 
There  are  many  ways  to  model  a  team.  In  our  approach  elicited  in  the  group  sessions  was  difficult  to  process 

we  distinguish  nine  so-called  team  modelling  afterwards.  Because  the  models  are  strongly  interrelated, 

perspectives  (Essens  et  al.,  2000).  These  are  an  adapting  them  became  a  major  effort.  For  future 

organisational  model,  a  function  model,  an  information  modelling  efforts,  further  technical  support  is  needed  for 

model,  a  function-information  model,  a  human  agent  consistency  checking,  glossary  update,  version 

model,  a  technical  means  model,  a  knowledge  model,  a  management,  and  for  having  navigational  overview 

task  model,  and  an  event  handling  model  (See  figure  2).  within  and  between  models.  Overall,  this  provided  us 

The  organisation  model  shows  the  relation  of  the  team  with  approximately  400  interrelated  graphical 

within  the  broader  organisation.  The  function  model  representations  implemented  with  a  browser  on  CD- 

shows  the  command  centre  functions  hierarchically.  The  ROM. 

information  model  shows  the  information  that  is  used 
and  processed  in  the  command  centre  hierarchically.  The  Techniques 
function-information  model  shows  the  information 

dependencies  of  the  functions  on  the  different  The  first  technique  applied  in  modelling  is  document 

hierarchical  levels.  The  agent  model  describes  the  team  analysis.  We  gathered  various  documents,  built  a 

member  roles;  the  technical  means  model  the  equipment  glossary  of  the  terminology  found,  and,  after  having 

that  team  members  have  to  their  disposal.  The  task  studied  the  material,  we  started  with  representing  our 

model  takes  the  end  nodes  of  the  functional  hierarchy  own  mental  model  of  the  human-human  system.  Next, 

and  couples  these  with  the  end  nodes  of  the  information  we  developed  a  visualisation  technique  to  describe  this 

hierarchy  (as  input  and  output),  the  technical  means,  mental  model,  first  for  communication  among  the 

agents,  and  the  knowledge  entities  as  controls.  Finally,  researchers.  We  used  a  representation  language  and 

the  event-handling  model  shows  the  response  to  a  graphics  in  the  models  consisting  of  a  restricted  set  of 

tactical  event  in  a  temporal  sequence  of  functions  that  descriptors  with  a  consistent  form  and  a  consistent 

are  distributed  over  agents.  meaning.  For  example,  in  figure  3,  an  arrow  means  data 


Tasks  are  fulfilled  within  Tasks  fulfill  Tasks  need  and  yield 

an  organisation  functions  information 


Tasks  are  performed  with  Tasks  '  Tasks  are 

knowledge  process  events  co-ordinated 


Figure  2:  Suite  of  models  for  the  description  of  a  complex  system 
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dependency,  a  small-circle  with  a  line  means  a  part-of 
relationship,  a  rounded  box  represents  a  function  or  task, 
a  square  box  represents  an  information  entity,  and 
knowledge  is  represented  by  a  square  box  with  a  line, 
After  this  preliminary  description,  we  used  the  graphical 
model  as  a  means  for  interviewing  the  domain  experts. 
Because  many  subjective  perspectives  arose  with  the 
various  experts  we  interviewed,  we  finally  asked  other, 
senior,  experts,  to  give  the  models  an  official  status,  a 
technique  we  called  standardisation. 


4  OBSERVATION 

Observations  are  needed  to  give  an  answer  to  the 
questions  that  arise  out  of  problems  that  are  experienced 
in  practice,  but  were  one  cannot  identify  the  specific 
bottlenecks  or  the  seriousness  of  these  bottlenecks.  It 
concerns  an  issue  description  in  order  to  develop  insight 
in  the  challenges  to  improve  team  effectiveness.  With 
observation  we  mean  a  systematic  analysis  and  on-line 
measurements  of  the  ongoing  process  of  team 


Figure  3:  example  of  one  level  of  the  function-information  model 


Figure  3  shows  an  example  of  the  used  representation 
language  for  the  function-information  model.  This 
example  describes  Command  &  Control  at  a  generic 
level  (Passenier  &  van  Delft,  1997).  The  functions  are 
linked  with  their  information  input  and  output  entities. 
The  diagram  shows  the  data  dependencies  between  the 
functions.  Situation  awareness  consists  of  a  permanent 
process  of  compilation,  monitoring,  and  maintenance  of 
the  actual  situation.  Outside  information  and  internally 
perceived  information  (i.e.,  from  own  sensors)  is  used  to 
develop  and  maintain  an  actual  situation  description. 
This  information  is  used  for  threat  assessment  by 
interpreting  the  current  situation  from  a  tactical 
perspective.  Planning  &  decision-making  is  performed 
by  selecting  options  and  planning  (counter)measures. 
The  decision  is  executed  during  direction  &  control  by 
giving  directives  and  allocating  resources. 


functioning  in  its  natural  setting.  This  provide  insights  in 
the  effectiveness  of  the  command  centre  team  and  reveal 
possible  bottlenecks  concerning  individual  task 
performance,  communications  and  information 
exchange,  co-ordination,  workload  and  so  forth.  This 
insight  is  needed  to  solve  the  experienced  problems 
either  by  designing  a  new  system  or  by  adjusting  the 
current  system. 

Project 

Observations  are  performed  to  investigate  the  command 
centre  of  the  RNLN  multipurpose  frigate  (see,  for  an 
elaborate  description,  Essens,  this  volume).  Four 
operational  command  centre  teams  performed  three 
realistic  warfare  scenarios  (e.g.,  baseline  normal 
pressure,  time  pressure  from  handling  multiple  objects, 
uncertainty  form  object  behaviours)  in  a  simulated 
command  centre  trainer  with  a  duration  of  three  hours 
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including  operational  briefing  at  start  shift.  Seven 
domain  experts  strategically  distributed  over  the  team 
members’  positions  determined  systematically  the 
performance  quality.  During  task  execution  each  team 
member  produced  a  workload  score.  A  questionnaire 
completed  data  collection.  In  a  debrief  session  a 
selection  of  the  observations  were  discussed  with  the 
experts,  the  investigators,  and  team  members  to  get  more 
insight  in  the  background  of  particular  actions  and 
decisions.  Based  on  these  observations  it  was  concluded 
that  in  times  of  stress  own  task  responsibilities  dominate 
and  push  away  teamwork  responsibilities  despite  the 
generally  strong  emphasis  on  total  team  performance. 

Techniques 

To  observe  systematically,  three  techniques  are 
developed  to  measure  performance:  a  quality  indicator 
sheet ,  a  workload  registration  tool ,  and  a  task- 
performance  questionnaire.  In  addition,  task 
performance  is  registered  digitally  on  video  (including 
audio)  that  is  used  for  reviewing. 

After  a  pre -briefing  about  the  content  of  the  research, 
performance  is  measured  during  task  execution.  Domain 
experts  (e.g.,  operational  experienced  officers)  score  the 
quality  indicator  sheet.  Each  time  a  problem  is  scored, 
the  experts  also  push  a  button  that  results  in  a  time 
registration  linked  to  the  video  stream.  This  is  used 
during  the  review  to  the  find  easily  the  video  data  that  is 
linked  to  the  scored  problem.  Workload  is  measured 
with  the  workload  registration  tool.  This  specially 
designed  button  panel  consists  of  five  buttons.  The 
buttons  correspondent  with  a  five  point  scale  that  each 
measures  a  degree  of  subjective  workload.  The 
registration  of  the  subjective  workload  takes  place  every 
five  minutes.  The  button  panels  are  linked  to  the  video 
stream  that  registers  the  measurements. 

Afterwards,  the  team  members  fill  in  the  task- 
performance  questionnaire.  In  the  meantime,  the  domain 
experts  and  the  researchers  review  and  discuss  the  task 
performance  based  on  the  scores  on  the  quality  indicator 
sheet.  During  this  review  it  is  determined  which 
observed  problems  are  the  most  critical  and  need  to  be 
made  more  explicit.  Subsequently,  the  domain  experts, 
the  researchers,  and  the  team  members  discuss  these 
problems  to  develop  a  deeper  understanding  of  the 
underlying  causes.  When  needed,  the  video  data  are 
showed  to  explicate.  All  together  the  experts’  scores,  the 
workload  measures,  the  questionnaires,  and  the 
discussion  reports  form  the  data  from  which  conclusions 
about  the  observed  problems  are  drawn. 

5  EXPERIMENTATION 

Observation  yields  insight  in  the  composite  set  of  factors 
that  influence  the  effectiveness  of  the  command  centre 
team.  From  observations,  however,  it  is  not  clear  to  what 
extent  single  factors  may  affect  team  effectiveness. 


Therefore,  we  investigate  single  factors  systematically 
by  using  a  contrived  experimental  task  in  the  form  of  a 
low  fidelity  simulator.  With  the  help  of  such  a  task  we 
attempt  to  identify  factors ,  and  develop  an  understanding 
of  their  effects  and  interactions.  There  are  several 
advantages  for  using  a  low-fidelity  simulation  (see  also, 
Bowers,  Salas,  Prince  &  Brannick,  1992).  First,  the 
methodology  is  available  at  relatively  low  cost.  Second, 
it  gives  opportunities  to  develop  the  characteristics 
needed  in  team  research.  Third,  it  provides  the  requisite 
experimental  control  of  independent  variables.  Finally,  it 
is  relatively  simple  to  train  team  members,  which  makes 
it  possible  to  invite  naive  participants  in  stead  of 
operational  team  members  that  are  difficult  to  recruit. 

Project 

Experimentation  is  used  to  investigate  the  effect  of  intra 
team  feedback  on  developing  shared  mental  models  in 
Command  &  Control  teams  (Rasker,  Post  &  Schraagen, 
in  press).  Team  members  that  share  mental  models,  are 
expected  to  have  common  expectations  of  the  task  and 
the  team,  allowing  them  to  predict  the  information  needs 
of  team  members  accurately.  This  makes  it  possible  for 
team  members  to  offer  each  other  the  necessary 
information  at  the  right  moment,  which  results  in  an 
improved  performance.  It  is  hypothesised  that  intra  team 
feedback  plays  an  important  role  in  adjusting  and 
developing  a  shared  mental  model.  By  giving  each  other 
feedback,  team  members  develop  an  understanding  of 
each  other’s  tasks  that  gives  them  insights  in  which  and 
when  information  must  be  exchanged.  A  distinction  is 
made  between  performance  monitoring  and  team  self- 
correction.  Performance  monitoring  is  the  ability  of  team 
members  to  monitor  each  other’s  task  execution  and  give 
feedback  during  task  execution.  Team  self-correction  is 
the  process  in  which  team  members  engage  in  evaluating 
their  performance  and  determining  their  strategies  after 
task  execution.  In  two  experiments,  the  opportunity  to 
engage  in  performance  monitoring  respectively  team 
self-correction  was  varied  systematically.  The  results 
show  that  teams  perform  better  when  they  have  the 
opportunity  to  engage  in  intra  team  feedback.  Both 
performance  monitoring  as  well  as  team  self-correction 
appeared  to  be  beneficial  for  team  performance. 

Techniques 

One  complicated  factor  of  studying  teams  using  a  low- 
fidelity  simulator  is  that  the  generalizability  to  real- 
world  environments  is  limited.  We  tried  to  reconcile  this 
by  developing  an  experimental  team  task  that  contains 
the  activities,  processes  and  situations  that  are  typical  for 
Command  &  Control  teams.  Based  on  a  generic 
Command  &  Control  model  we  performed  a  task 
analysis  using  the  previously  described  modelling 
method  that  provides  not  only  a  task  hierarchy  but  also 
describes  the  information  dependency  between  tasks,  the 
knowledge  needed  to  perform  tasks  accurately,  and  the 
sequence  of  tasks  for  each  team  member.  Based  on  this 
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task  analysis  we  specified  the  different  roles  and 
expertise  of  team  members  and  the  information 
dependency  between  them.  In  addition,  by  showing  that 
the  specified  tasks  have  to  be  performed  in  parallel,  we 
demonstrated  that  the  experimental  task  is  a  task  for  two 
team  members,  which  cannot  be  performed  individually. 
Based  on  the  task  analysis,  we  attempted  to  develop  an 
experimental  team  task  that  contains  the  advantages  of 
low  fidelity  simulations  but  still  can  be  generalised  to 
real-world  environments. 

The  experimental  task  we  developed  is  a  low-fidelity 
simulation  of  a  dispatch  centre  representing  a  fire¬ 
fighting  organisation  in  a  city  (Rasker,  Post  & 
Schraagen,  in  press).  The  fire-fighting  team,  consisting 
of  an  observer  and  an  allocator ,  is  required  to  fight  fires 
in  order  to  keep  the  number  of  casualties  as  low  as 
possible,  which  is  the  goal  of  the  task.  In  order  to 
accomplish  the  goal,  the  observer  has  to  assess  the  city 
and  inform  the  allocator  about  the  status  of  the  buildings. 
This  is  comparable  to  the  situation  assessment  function 
in  Command  &  Control.  The  allocator  has  to  allocate  a 
number  of  resources  (i.e.,  fire-fighting  units)  to  the 
buildings  to  extinguish  fires.  This  is  comparable  to  the 
function  of  decision-making  and  taking  counter 
measures  in  Command  &  Control.  Because  the  number 
of  units  is  limited,  the  team  must  prioritise  and  decide 
upon  which  fires  they  want  to  extinguish.  Team 
members  are  interdependent  of  each  other.  This  means 
that  they  are  required  to  interact  continuously  about  the 
status  of  fires  and  the  resources  to  accomplish  the  goal. 

6  DESIGN 

One  part  of  the  development  of  a  new  human-human- 
machine  system,  such  as  a  new  platform,  is  the  design  of 
certain  team  organisations  such  as  for  Command  & 
Control,  navigation,  and  ship  control.  Team  design  can 
be  at  a  conceptual  level  (e.g.,  a  proposed  task 
organisation)  or  a  can  be  a  realisation  at  a  detailed  level 
(e.g.,  the  layout  of  a  command  centre).  A  team 
organisation  may  be  redesigned,  when  bottlenecks  or 
challenges  are  observed  in  an  existing  team.  In  a  design 
process,  the  known  team  factors  are  applied  to  new  team 
concepts  and  actual  working  teams. 

In  designing  a  human-human  system  the  following 
phases  can  be  differentiated  (Beevis,  Essens  and 
Schuffel,  1995): 

•  Defining  the  mission  of  the  overall  system 

•  Establishing  the  functions  that  need  to  be  fulfilled 

•  Determining  the  tasks  that  need  to  be  carried  out 

•  Determining  the  required  manning  and  means 

•  Job  design  (task  allocation) 

•  Workplace  design 

Project 

The  design  process  is  illustrated  by  the  development  of 
the  command  centre  of  the  Air  Defence  and  Command 


Frigate  of  the  RNLN.  The  first  phases  of  the  design 
process  were  already  established  by  the  RNLN:  the 
mission  to  establish  the  functions  to  fulfil  the  number  of 
agents  needed,  the  choice  of  equipment,  and  job  design. 
Our  role  in  the  design  team  was  to  design  the  workplace 
(Post  &  Punte,  1997;  Post  &  Punte,  1998). 

From  the  literature  and  our  own  research,  a  number  of 
factors  for  team  effectiveness  were  identified: 

•  shared  mental  model  (Cannon-Bowers,  Salas  & 
Converse,  1993) 

•  non-verbal  communication  (Fussel  &  Benimoff,  1995) 

•  adaptive  co-ordination  (Entin  &  Serfaty,  1999) 

•  situation  awareness  (Endsley,  1995) 

•  intra  team  feedback  (Rasker  et  al.,  in  press) 

•  team  training  (Salas  et  al.,  1993) 

From  these  factors,  and  the  design  decisions  made  by  the 
RNLN  in  the  earlier  phases,  two  concepts  on  team 
support  (as  well  as  other  aspects,  such  as  technical  and 
economical  feasibility)  were  generated  and  assessed. 
Both  concepts  had  a  number  of  aspects  in  common  that 
already  were  an  improvement  to  the  most  recent 
command  centre  that  was  designed  for  the  RNLN,  such 
as  the  use  of  large  screen  displays  for  deliberation 
support,  and  a  briefing  room  for  training  support.  One 
concept  was  a  traditional  one,  based  on  the  most  recent 
command  centre  that  was  designed:  consoles  placed  in 
three  rows,  the  middle  one  the  command  row,  the  row  in 
front  the  picture  compilation  and  weapon  employment 
row,  and  the  row  in  the  back  the  support  row  (e.g., 
communication  support,  equipment  support,  network 
support).  In  the  other  concept,  the  traditional  way  was 
left  and  the  rows  were  placed  in  a  circle.  We  found  that 
in  a  circle,  the  mean  distance  between  team  members 
was  shortest. 

The  second  rounded  concept  proved  to  be  the  best  at  the 
aspect  of  team  support.  For  individual  work  conditions, 
however,  the  rounded  concept  was  unacceptable  due  to 
adverse  ship  motion  effects  (Bittner  &  Guignard,  1985). 
Nonetheless,  a  number  of  principles  for  this  concept 
were  taken  over.  This  led  to  the  final  conceptual  design, 
with  large  screen  displays  and  an  integrated  briefing 
room,  that  was  optimised  for  frequent  and  critical 
interactions  (i.e.,  putting  those  people  near  each  other 
who  need  to  work  in  closest  co-operation).  Further, 
visual  deliberation  and  supervision  lines  at  larger 
distances  could  be  optimised  by  placing  the  individual 
work  places  nor  in  a  row,  neither  in  a  circle,  but  in  a  kind 
of  wave  form. 

The  next  step  was  to  translate  the  conceptual  design  in  a 
detailed  design.  At  this  stage,  we  made  a  full  scale 
wooden  mock  up  of  both  the  individual  work  places  and 
the  complete  command  centre,  and  invited  in  total  75 
future  users  to  assess  the  design  (among  other  aspects) 
on  co-operation,  deliberation  and  supervision  support. 
Based  on  this  finding  the  command  centre  was 
redesigned  and  approved  for  realisation. 
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Techniques 

In  designing  the  workplace,  we  used  a  number  of 
techniques: 

•  link  analysis 

•  drawings 

•  scenarios 

•  physical  mock-ups 

In  RNLN  example,  we  only  designed  the  workplace  of 
the  command  centre.  Currently  we  are  carrying  out 
research  for  the  RNLN  future  Command  &  Control,  in 
which  we  also  design  reduced  command  centre  teams, 
including  team  size,  necessary  means,  and  job  design. 

7  EVALUATION 

The  ultimate  goal  of  modelling,  observation, 
experimentation  and  design  is  improving  the 
effectiveness  of  team  performance.  Evaluation  stands  for 
determining  and  verifying  this  effectiveness.  Two  types 
of  measures  are  important  for  team  effectiveness: 
process  measures  and  outcome  measures.  Process 
measures  refer  to  speed  and  workload ,  and  outcome 
measures  to  quality  and  quantity. 

Project 

Evaluation  is  not  only  possible  at  a  final  stage,  when  a 
design  has  been  realised.  In  the  command  centre 
example  provided  above,  we  used  a  wooden  mock  up  for 
evaluation  purposes,  and  had  RNLN  personnel  perform  a 
scenario.  We  asked  them  to  fill  in  a  questionnaire  on 
team  aspects.  The  questions  were  about  the  positioning 
of  their  co-workers,  how  the  command  team  was  able  to 
deliberate,  how  supervisors  could  view  their  personnel 
from  their  positions,  and  how  they  could  communicate 
non-verbally.  We  let  them  compare  seven  row 
configurations  (such  as  a  straight  row,  an  indented  row 
and  a  waved  row)  on  the  team  aspects  mentioned  above. 
Moreover,  we  made  this  comparison  for  three  different 
individual  work  places:  one  with  a  single  screen,  one 
with  two  20"  screen,  and  one  with  a  primary  20"  screen 
and  a  secondary  14"  screen,  making  up  21  (7  times  3) 
configurations.  We  could  find  at  statistical  significance 
level  how  type  of  row  and  type  of  individual  work  place 
relate  to  team  aspects.  Figure  4  shows  some  of  the 
configurations  of  rows. 

Another  evaluation  example  is  obtained  from  the 
SmartStaff  project  (see,  for  an  elaborate  description,  Post 
&  Hamaker,  this  volume).  The  aim  of  this  project  is  to 
find  the  best  team  support  for  the  Task  Group  Staff  This 
staff  is  embarked  on  board  of  the  Air  Defence  and 
Command  Frigate,  and  has  its  own  Staff  room.  In 
contrast  to  the  command  centre  and  the  navigation 
bridge  of  this  frigate,  the  RNLN  had  no  clear  vision  on 
how  to  equip  the  staff  room.  Therefore,  we  took  it  as  a 
research  challenge  to  come  up  with  a  support  concept  for 
team  planning,  which  is  the  main  task  of  the  Task  Group 


Staff.  The  resulting  concept,  called  Smartstaff,  supports 
the  team  with: 

•  both  individual  and  shared  work  spaces 

•  flow  and  storage  of  electronic  information 

•  a  common  focus  of  attention 

•  concurrent  idea  generation 


Figure  4:  Configurations  of  rows.  The  arrows  show  some 
lines  of  views 

SmartStaff  has  been  evaluated  by  using  a  staff  room 
mock-up  with  for  each  team  member  an  individual 
workstation  and  two  large  interactive  touch  screen 
displays,  electronic  storage,  retrieval  and  exchange  of 
information,  and  an  electronic  idea  pad:  a  tool  for  both 
individual  and  team  generation  and  representation  of 
ideas.  To  guarantee  face-to-face  contact  and  an  unlimited 
view  of  the  large  screen  display,  the  workstations  were 
lowered  and  positioned  in  a  semi-circle  around  the  large 
screen  display. 

Our  aim  was  to  carry  out  a  conceptual  evaluation.  At  an 
early  stage,  we  wanted  to  know  whether  our  concept  was 
profitable,  without  having  to  implement  the  concept 
fully.  We  were  able  to  reduce  the  software  development 
effort  to  a  minimum,  by  using  existing  software  as  much 
as  possible,  and  letting  the  integration  efforts  (often  the 
most  expensive  parts  of  software  development)  over  to 
the  evaluators,  who  had  therefore  to  strike  some  extra 
keys  during  the  evaluation  session. 

In  this  evaluation  session,  all  14  RNLN  Task  Group 
Staff  members,  including  the  Commodore,  served  as 
subjects.  They  were  first  asked  to  fill  in  a  questionnaire 
about  their  current  staff  room.  Next,  they  had  a  one-hour 
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training  in  using  the  SmartStaff  environment.  Then,  they 
were  provided  with  a  realistic  scenario  in  which  they  had 
to  plan  a  journey  for  the  Task  Group.  After  that,  they  had 
to  fill  in  again  a  questionnaire,  but  now  about 
SmartStaff.  Having  experienced  this  new  task 
environment,  the  evaluation  session  concluded  with  a 
group  discussion.  During  the  session,  an  external 
observer,  who  joined  the  Staff  earlier,  assessed  the  team 
as  well.  It  was  found  that  SmartStaff  provides  better 
support  on  the  unambiguity  of  the  shared  picture  of  a 
tactical  situation  and  the  plan  itself,  on  time 
management,  and  on  presenting  and  communications  of 
ideas  and  plans.  The  quality  of  the  plan  was  not 
influenced. 

Techniques 

Summarising,  the  techniques  used  in  both  studies  are: 

•  hardware  and  software  mock-ups 

•  Questionnaires 

•  Scenarios 

•  External  assessor 

•  Experienced  group  discussion 

8  SUMMARY  &  CONLUSIONS 

Effective  teamwork  is  a  prerequisite  in  a  complex  system 
such  as  the  command  centre.  Investigating  teamwork  and 
determining  what  makes  a  team  effective  is  a  great 
challenge  for  team  researchers.  The  objective  of  this 
paper  is  to  describe  in  a  framework  how  several  research 
methods  can  be  applied  to  investigate  command  centre 
teams.  This  framework  helps  to  place  the  questions  that 
come  from  team  practice  in  the  set  of  methods  and 
techniques  that  are  currently  available.  The  research 
framework  that  is  proposed  comprises  five  methods  that 
cover  all  aspects  of  team  research:  modelling, 
observation,  experimentation,  design  and  evaluation.  For 
each  method,  the  research  framework  was  illustrated 
with  a  number  of  projects  and  the  techniques  Table  1 
gives  an  overview  of  the  methods,  matched  with  the 


questions  for  the  specific  projects,  the  used  techniques, 
and  the  results. 

Matching  the  questions  to  the  methods  makes  it  clear,  in 
our  opinion,  which  research  effort  is  possible  to  provide 
usable  answers.  This  serves  as  a  guidance  to  make  a 
choice  between  the  methods  available.  Although  the 
methods  are  treated  separately,  they  are  related  with  each 
other.  Most  important  is  that  they  all  contribute  to  our 
knowledge  of  how  teams  perform  and  which  factors 
make  teams  effective.  This  knowledge  is  used  to  modify 
and  refine  the  research  techniques  and  give  direction  in 
our  new  research  efforts.  Moreover,  this  knowledge  is 
used  to  develop  new  concepts  and  realisations  for 
optimising  team  effectiveness.  We  conclude  that  the 
research  framework  provides  a  useful  overview  of  the 
methods  one  can  apply  to  investigate  command  centre 
teams.  Each  method  can  be  used  to  find  an  answer  to  a 
part  of  the  puzzle  how  to  optimise  effective  teamwork. 
In  total,  the  research  framework  represents  an  integral 
approach  for  investigating  command  centres  teams  that 
should  lead  to  an  answer  of  the  complete  puzzle. 

For  future  research  concerning  team  effectiveness  in  the 
command  centre,  all  methods  will  be  applied.  Modelling 
will  be  applied  to  describe  the  command  centre  of  the 
new  mine  counter  measure  vessels  of  the  RNLN.  In 
addition,  we  are  planning  to  elaborate  our  modelling 
techniques  in  order  to  simulate  team  behaviour  in 
Command  &  Control  situations.  This  way  we  are  able  to 
use  modelling  not  only  for  description  but  also  for 
experimentation.  Different  factors  will  be  modelled  in 
order  to  investigate  their  effects  on  team  effectiveness. 
Furthermore,  we  will  design  a  concept  for  improving  the 
briefing  sessions  to  achieve  more  effective 
communication  in  command  centre  teams.  This  design 
will  also  be  evaluated.  With  this  and  other  research 
efforts,  we  attempt  to  develop  a  new  concept  for  future 
Command  &  Control  that  improves  team  effectiveness  in 
the  command  centre. 


Modelling 

Observation 

Experimentation 

Design 

Evaluation 

Project 

Multipurpose  frigate 

Multipurpose  frigate 

Intra  team  feedback 

Air  Command  &  Defence 
frigate 

Air  Command  &  Defence  frigate 
SmartStaff 

Question 

What  is  the 

information  flow  in  the 
command  centre? 

How  can  the 
performance  of  the 
command  centre  be 
optimised? 

What  is  the  effect  of 
intra  team  feedback  on 
developing  shared 
mental  models? 

What  is  the  best  layout  of 
the  command  centre  to 
support  effective  human- 
human  interaction? 

Does  the  design  fulfil  its’ 
expected  effectiveness? 

Techniques 

-  document  analyses 
-visualisation 

-  interviewing 

-  standardisation 

-  quality  indicator  sheet 

-  workload  registration 
tool 

-task  performance 
questionnaire 

Low  fidelity  simulator: 
fire-fighting  task 

-  link  analysis 

-  drawings 

-  scenarios 

-  physical  mock-ups 

-  hardware  and  software  mock- 
ups 

-  questionnaires 

-  scenarios 

-  external  assessor 

-  experienced  group  discussion 

Result 

clear  description  of  the 
command  centre  with 
400  interrelated 
graphical 
representations 

individual  task 
responsibilities 
dominate  teamwork 
responsibilities 

teams  perform  better 
when  they  have  the 
opportunity  to  engage 
in  intra  team  feedback 

complete  workplace  layout 
supporting  team 
effectiveness 

-  workplace  layout  mock-up 

-  SmartStaff  concept  supporting 
collaborative  decision-making 
and  planning 

Table  1:  overview  of  the  methods,  projects,  questions,  techniques  and  results 
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SUMMARY 

A  new  concept  has  been  evaluated  to  support  decision 
making  in  teams.  The  concept  encompasses  a  shared 
representation  and  interactive  use  of  planning 
information  in  a  team  environment,  and  consists  of 
individual  workplaces,  generation  and  representation  of 
ideas,  and  shared  interactive  large  screen  displays.  This 
so-called  SmartStaff  concept  has  been  evaluated  during  a 
simulated  operation  by  the  Task  Group  Staff  of  the 
Royal  Netherlands  Navy.  By  means  of  questionnaires  the 
staff  members  were  asked  to  assess  their  current  work 
environment  and  the  potentials  of  SmartStaff.  The  results 
show  that  the  concept  provides  better  general  support  for 
group  decision  making.  SmartStaff  supported  better  the 
presentation  and  conveyance  of  ideas,  facilitated  time 
management  and  decreased  the  ambiguities  of  the  plans 
presented.  However,  the  quality  of  the  final  plan  did  not 
improve. 

KEYWORDS 

Team  decision  making,  Team  Planning,  Group  Support 
Systems,  Task  Group  Staff 

1  INTRODUCTION 

A  concept  for  supporting  a  planning  task  in  a  team  has 
been  evaluated.  The  goal  of  this  so-called  SmartStaff 
concept  is  to  support  a  team  in  developing  a  common 
representation  of  both  the  problem  space  (an  operational 
situation)  and  the  solution  space  (a  plan  to  be 
developed).  This  concept  has  been  implemented  in  an 
environment  where  planners  can  work  both  individually 
and  together,  while  using  and  producing  information  in  a 
highly  interactive  way. 

A  number  of  researchers  employ  the  term  ‘shared 
mental  model’  in  explaining  effective  team  behaviour 
(e.g.,  Orasanu,  1990,  Rouse,  Cannon-Bowers  &  Salas, 
1992,  Stout,  1995).  The  SmartStaff  concept  encompasses 
a  representation  of  a  shared  mental  model  of  the 
operational  situation.  SmartStaff  also  includes  a 
representation  of  a  shared  mental  model  of  the  plan  that 
the  team  is  developing. 

To  improve  our  understanding  of  team  support,  our  aim 
is  not  to  evaluate  a  particular  environment  but  rather  the 
concept  behind  it.  In  order  to  evaluate  the  concept 


empirically,  we  have  implemented  the  concept  in  a  naval 
command  and  control  task  of  the  Task  Group  Staff  of  the 
Royal  Netherlands  Navy  (RNLN)  that  has  operational 
command  over  a  number  of  naval  platforms. 

Supporting  naval  command  and  control  teams  has  been 
the  subject  of  a  large  research  program  called  TADMUS 
(Tactical  Decision  Making  Under  Stress);  see  Cannon- 
Bowers  and  Salas  (1998)  for  an  overview.  Morisson, 
Kelly,  Moore  and  Hutchins  (1998)  discuss  a  number  of 
decision  support  systems  developed  in  this  program,. 
The  design  of  these  systems  were  based  on  the 
naturalistic  decision  making  theory  (Zsambok  and  Klein, 
1997),  stating  that  a  decision  is  most  of  the  time  based 
on  a  recognition  of  a  previously  experienced  pattern. 
Decision  support  should  therefore  facilitate  the 
recognition  of  these  patterns.  Example  decision  support 
systems  designed  by  Morisson  et  al.  are  a  geo-plot  (a 
computer  graphic  representation  of  a  geographic  area 
with  associated  information,  i.e.  land  masses,  political 
boundaries,  symbols  for  assets  and  units),  and  a  track 
profile  (graphically  displaying  the  altitude  of  an  air 
contact  over  time  and  range  from  own  ship).  However, 
supporting  team  planning  has  not  been  part  of  the 
research  program. 

A  new  conceptual  approach  is  an  important  aspect  of  the 
development  of  a  new  command  frigate,  the  platform 
that  embarks  the  Task  Group  Staff.  Traditionally,  the 
RNLN  designs  its  own  frigates.  Since  the  start  of 
operation  of  the  current  command  frigate,  three  decades 
ago,  much  has  changed  in  the  field  of  information  and 
communication  technology.  New  ways  of  working  have 
to  be  designed,  updated  not  only  to  the  current  state  of 
technology  but  also  to  be  prepared  for  developments  in 
the  future.  Therefore,  a  conceptual  evaluation  is  more 
valuable  than  merely  a  state-of-the-art  based 
technological  evaluation. 

1.1  Technological  Development 

A  most  significant  change  in  the  past  decades  is  the  use 
of  electronic  information  in  groups.  McGrath  and 
Hollingshead  (1994)  give  three  reasons  why  teams 
should  work  with  electronic  information: 

•  it  can  improve  task  performance, 

•  it  can  overcome  time  and  space  constraints,  and 

•  it  enhances  information  retrieval  and  exchange. 


Paper  presented  at  the  RTO  HFM  Symposium  on  “  Usability  of  Information  in  Battle  Management 
Operations”,  held  in  Oslo ,  Norway ,  10-13  April  2000,  and  published  in  RTO  MP-57. 
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Important  recent  developments  in  command  and  control 
are  the  paperless  ship,  the  large  screen  displays,  and 
various  electronic  support  tools.  Electronic  support  tools 
can  be  categorised  in  different  ways.  Group  support 
systems  can  be  distinguished  on  the  basis  of  time  and 
space  constraints  (DeSanctis  and  Gallupe,  1987, 
Grudin,  1997).  People  can  work  together  at  the  same 
place  or  at  different  locations,  and  also  work  together 
synchronously  or  a-synchronously.  A  second  distinction 
of  group  support  systems  is  type  of  task.  Computer 
supported  cooperative  work  can  be  divided  in: 

•  communication  between  co-workers; 

•  creating  and  maintaining  a  shared  information 
space;  and 

•  coordination  of  the  various  interactions  between  the 
co-workers,  and  between  a  worker  and  the 
information  system. 

Group  systems  that  support  communication  mainly  deal 
with  groups  that  are  distributed  in  space,  some  of  which 
work  synchronously  (telephone,  video  conferencing), 
and  others  a-synchronously  (e.g.,  e-mail).  Desktop 
conferencing  is  an  example,  given  by  Grudin  (1997),  of 
a  type  of  support  system  that  creates  and  maintains  a 
shared  information  environment  in  which  group 
members  can  share  large  screen  displays  and  different 
electronic  tools.  There  are  several  group  decision  support 
system,  for  example  computer  conferencing  tools, 
application  sharing  systems,  collaborative  virtual 
environments,  audio  conferencing  systems,  and 
collaborative  software  engineering  systems  (Grudin, 
1997,  Ter  Hofte,  1998). 

A  rather  new  device  that  can  support  group  work  is 
the  shared  electronic  whiteboard.  Originally,  shared 
whiteboards  were  used  for  groups  working  at  different 
locations  who  need  to  work  on  a  common  object  (e.g.,  a 
document)  during  video  conferencing,  but  it  has  been 
found  to  be  useful  in  face-to-face  meetings  too.  Streitz, 
Geissler,  Haake  &  Hoi  (1994),  for  example,  compared 
three  conferencing  configurations.  In  one  configuration, 
a  group  of  graphical  designers  were  provided  with 
individual  workstations.  In  a  second  configuration,  an 
interactive  whiteboard  was  provided.  The  third 
configuration  consisted  of  the  mixture  of  both.  They 
found  that  designers  supported  by  both  individual 
workstations  and  an  interactive  whiteboard  performed 
best,  in  terms  of  quantity  and  quality  of  ideas,  amount  of 
activity,  and  a  shared  picture  of  the  subjects  of 
discussion.  The  whiteboard  that  was  used  presented  a 
computer  screen  and  enabled  direct  interaction  or 
interaction  from  behind  the  individual  workstations.  The 
whiteboard  also  allows  drawing  pictures  with  an 
electronic  pen.  It  appears  that  this  concept  improves 
performance,  because  the  whiteboard  focuses  attention 
on  the  design  object  as  well  as  the  design  process,  and 
facilitates  the  comparisons  of  ideas  (plans);  while  idea 
generation  takes  place  interactively  at  the  whiteboard, 
other  members  can  respond  to  it  immediately. 


1.2  Support  Concept 

Planning  may  be  defined  as  designing  a  sequence  of 
actions  to  be  taken  in  order  to  react  upon  an  anticipated 
threat  with  regard  to  a  mission,  where  all  actions  are 
heavily  interdependent  and  where  design  decisions  need 
confirmed  arguments.  Team  planning  requires  that 
information  needed  for  planning  as  well  as  the  plan  itself 
is  shared  and  that,  for  efficiency  reasons,  team  members 
can  work  both  collectively  and  individually.  It  is 
essential  that  the  separate,  diverging  work  of  members 
having  individual  expertise  is  followed  by  convergence 
of  ideas  in  the  team. 

Current  electronic  tools  do  not  support  team  planning. 
With  an  electronic  conference  tool,  for  instance,  a  group 
can  first  generate  a  list  of  individual  ideas  and  next  come 
to  an  agreement  of  the  best  one.  For  staff  planning, 
however,  a  tool  is  needed  that  enables  staff  members  to 
develop  a  single  plan  or  ideas  for  plan  refinement  in  a 
collective  and  integrated  way  rather  than  enabling  only 
individual  disconnected  idea  generation.  Further,  we 
think  that  for  interactive  planning,  it  is  not  only 
important  to  share  planning  information  and  the  plan 
itself;  sharing  information  about  the  planning  process  is 
essential  as  well.  Information  about  the  planning  process 
gives  other  members  the  possibility  of  reacting 
immediately  during  planning.  They  may  contribute 
concurrently  instead  of  sequentially  (i.e.,  first  developing 
a  partial  plan  individually,  and  then  discussing  it  within 
the  team.) 

On  the  basis  of  these  ideas  the  SmartStaff  was 
conceptualized,  having  the  following  characteristics.  A 
planning  staff  needs: 

1 .  Both  individual  and  shared  workspaces 

2.  Flow  of  information 

3.  A  common  focus  of  attention 

4.  Concurrent  idea  generation 

An  “idea”  is  used  as  the  unit  for  information 
conveyance.  An  idea  can  take  various  forms:  text,  an 
object  (graphical,  but  potentially  also  audio),  or  even  a 
reference  link  to  another  idea.  Ideas  can  consists  of  sub¬ 
ideas.  A  plan  is  typically  a  compound  idea. 

We  have  implemented  this  concept  with  eight  individual 
workstations  and  two  large  interactive  touch  screen 
displays  (electronic  whiteboards),  electronic  storage, 
retrieval  and  exchange  of  information,  and  an  electronic 
idea  pad:  a  tool  for  both  individual  and  team  generation 
and  representation  of  ideas.  To  guarantee  face-to-face 
contact  and  an  unlimited  view  of  the  large  screen 
displays,  the  workstations  were  lowered  and  positioned 
in  a  semi-circle  around  the  large  screen  displays.  Figure 
1  shows  the  layout  of  the  experimental  staff  room. 

As  idea  pad  we  used  a  commercially  available  software 
tool  (SmartNotebook,  from  Smart  Technologies  Inc.), 
but  in  a  specific  way.  In  the  idea  pad,  ideas  may  be  put 
on  a  single  page,  or  divided  over  different  pages.  The 
development  of  an  idea  may  be  recorded,  enabling 
skipping  back  to  an  earlier  development  phase.  Elements 
of  an  idea  can  be  made  from  scratch  or  may  be  imported 
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from  existing  sources,  including  other  idea  pads  or  idea 
pad  pages.  Other  available  electronic  information 
sources  are  the  tactical  situation,  meteorological 
information,  charts,  mission  statement,  messages, 
intelligence,  etc. 


Figure  1 :  The  layout  of  the  experimental  staff  room. 

Ideas  can  be  exchanged  with  a  specially  developed  tool. 
When  a  team  member  wants  to  submit  an  idea  he  can 
send  his  idea  pad  to  a  common  idea  repository,  presented 
at  the  large  screen  display  with  the  name  of  the  sender, 
and  a  one-line  description.  The  content  of  the  private 
idea  pad  can  be  discussed  within  the  team,  and  accepted 
(possibly  after  revision  by  the  individual  or  the  team)  as 
(part  of)  a  shared  team  idea,  or  thrown  away.  The  other 


way  around,  a  shared  team  idea  can  be  fetched  from  the 
common  idea  repository  and  placed  on  the  individual’s 
own  work  space,  for  example  for  extension,  correction, 
or  refinement. 

1.3  Task  Analysis 

This  paper  describes  the  empirical  evaluation  of  the 
SmartStaff  concept  in  the  naval  command  and  control 
task  of  task  group  staff  planning.  The  aim  of  this  study  is 
to  examine  whether  the  team  decision  making 
performance  improves  when  the  team  is  supported  by  the 
SmartStaff  concept.  More  specifically,  we  want  to 
investigate  whether  SmartStaff  improves  a  shared  picture 
of  the  situation  and  the  plan,  is  more  time-efficient, 
improves  the  communication  of  ideas  and  the  quality  of 
the  final  plan. 

Before  the  evaluation  we  first  analysed  the  work  of  the 
Staff.  Naval  platforms  seldom  operate  individually,  but 
rather  in  a  group,  called  a  Task  Group.  The  Task  Group 
Commander,  supported  by  a  team  varying  from  five  to 
fifteen  members  with  a  specific  individual  expertise, 
such  as  in  meteorology,  intelligence,  communications, 
and  the  different  warfare  areas,  exercises  command  from 
a  dedicated  frigate.  Monitoring,  threat  assessment,  and 
control  of  operations  of  the  whole  Task  Group  takes 
place  in  the  staff  room  of  this  command  frigate.  Various 
phases  of  an  operational  situation  can  be  distinguished, 
differing  in  threat,  workload  and  intensity  of  the  team 
decision  making  process.  The  planning  process  is 
illustrated  in  figure  2. 


Figure  2:  Staf  tasks.  The  rounded  boxes  are  tasks;  the  shaded  boxes  are  information  boxes;  arrows  are  data 
dependencies 


5-4 


Based  on  its  mission,  the  Staff  plans  the  activities  of  the 
task  group  and  executes  the  plan  by  directing  the  units. 
During  execution,  both  the  situation  and  the  execution 
progress  is  monitored.  When  the  situation  changes  and 
problems  in  executing  the  plan  are  anticipated,  or  the 
mission  changes,  the  plan  is  revised.  This  process  is 
carried  out  in  three  cycles: 

1.  long-term  (re)planning  (more  than  half  a  day  in 
advance) 

2.  short-term  (re)planning  (up  to  half  a  day  in  advance) 

3.  near-real-time  decision  making 

During  these  cycles,  briefings  take  place  regularly  to 
inform  the  members  of  the  staff.  Re-planning  occurs  by 
generating  collectively  a  solution  in  a  rough  form,  and 
then  working  out  the  details  in  the  plan  individually,  and 
next,  discussing  them  collectively.  The  final  adaptations 
are  translated  into  orders  and  sent  to  the  task  group  units. 

In  its  current  environment,  the  Task  Group  Staff  does 
not  work  much  with  electronic  information.  The  Staff 
does  not  have  electronic  presentation  or  electronic  data 
exchange  facilities;  much  is  done  on  paper,  and  on  white 
boards  and  tote  boards. 

2  METHOD 

2.1  Subjects 

All  14  RNLN  Task  Group  Staff  members  served  as 
subjects,  6  Petty  Officers  1st  class,  6  1st  Lieutenants,  a 
Captain  and  a  Commodore.  The  1st  Lieutenants  were 
experts  in  one  or  more  particular  areas  (operations,  the 
three  warfare  areas,  meteorology,  communication,  and 
intelligence).  The  Captain  and  the  Commodore  were  the 
team  leaders.  The  Petty  Officers  supported  the  Officers. 
All  subjects  had  significant  operational  experience,  also 
within  this  team  (except  for  the  Captain).  Mean  age  was 
39  years  (34  to  50). 

2.2  Design 

The  RNLN  Task  Group  Staff  carried  out  their  work 
during  a  simulated  operation  in  the  SmartStaff-based 
environment.  For  pragmatic  reasons,  we  were  not  able  to 
make  a  pure  experimental  comparison  between  this 
environment  and  an  environment  not  based  on 
SmartStaff  .  Only  one  RNLN  Task  Group  Staff  exists 
and  its  time  is  restricted.  Therefore,  the  staff  members 
were  asked  to  compare  this  experimental  environment 
with  their  normal  working  environment.  To  make  their 
work  as  similar  as  possible,  we  used  an  operational 
scenario  that  was  comparable  with  a  training  scenario 
they  had  used  earlier  in  their  current  environment.  In 
addition  to  this  self  assessment,  we  invited  two  experts 
in  the  field  as  independent  observers,  to  collect 
information  for  the  interpretations  of  the  results. 

Three  questionnaires  were  developed:  one  for 
assessing  the  current  environment  (A),  one  for  assessing 
SmartStaff  (B),  and  one  for  comparing  directly  both 
environments  (C).  With  the  three  questionnaires, 


SmartStaff  was  tested  in  two  ways: 

•  Indirectly,  by  comparing  the  questions  about  the 
current  environment  (questionnaire  A)  with  the 
questions  about  SmartStaff  (questionnaire  B) 

•  Directly,  by  testing  the  null  hypothesis  that 
SmartStaff  and  the  current  environment  supports  the 
Task  Group  Staff  equally  well.  This  is  done  by 
comparing  the  questions  in  which  both 
environments  were  compared  (questionnaire  C) 
with  the  answer  ‘equal’. 

The  subjects  as  well  as  the  observers  filled  out  the  three 
questionnaires.  The  questions  in  each  questionnaire  were 
organised  in  5  modules: 

1.  point  of  focus  and  shared  picture  (3  questions),  e.g., 
“How  often  during  a  meeting  in  <the  current 
environment  do  you  have  a  different  picture  of  the 
situation  than  a  colleague?” 

2.  efficient  use  of  individual  and  shared  time  (5 
questions),  e.g.,  “How  often  during  <the  SmartStaff 
meeting>  do  you  experience  that  you  lose  time?” 

3.  communication  of  ideas  (5  questions),  e.g.,  “Can 
you  clarify  your  ideas  with  SmartStaff  better  than  in 
the  current  environment?. 

4.  product  quality  (1  question),  e.g.,  “How  do  you 
assess  the  mean  quality  of  the  plans  resulting  from 
the  <current  environment^’? 

5.  general  questions  (12  questions)  ,  e.g.,  “How  well 
can  you  present  your  ideas  to  the  team  in  <the 
current  environment?”;  “How  well  does 
<SmartStaff>  support  you  in  participating  in  the 
team  discussion?”. 

In  questionnaire  A  and  B,  answers  had  to  be  given  on  a 
4-point  scale  (bad,  rather  bad,  rather  good,  good).  In 
questionnaire  C,  a  5-point  Likert  scale  was  used  (with 
answer  categories  ‘much  worse’,  ‘worse’,  ‘equal’, 
‘better’,  ‘much  better’).  This  questionnaire  also  asked 
some  open  question  about  both  environments,  such  as 
about  their  strengths  and  weaknesses,  possible 
improvements,  and  the  potential  of  SmartStaff,  e.g.,  “Do 
you  have  any  suggestion  for  improving  SmartStaff?” 

The  subjects  as  well  as  the  observers  also  took  part  in  a 
group  discussion,  taking  place  after  having  experienced 
the  SmartStaff  environment.  The  group  discussion  was 
also  based  on  qualitative  questions. 

2.3  Scenarios 

A  realistic  simulation  in  the  environment  described 
above  requires  a  full  scenario  in  which  the  Staff  directs  a 
Task  Group  consisting  of  various  frigates,  tankers,  an 
amphibious  unit,  air  units,  and  a  submarine,  within 
political  constraints  laid  down  in  so  called  Rules  of 
Engagement.  The  scenario  was  based  on  a  training 
scenario,  adapted  to  the  above  mentioned  three  team 
decision  making  cycles. 

For  long-term  planning,  the  task  was  to  prepare  a  plan  to 
escort  a  Task  Unit  to  a  particular  waiting  area  prior  to  an 
amphibious  landing  by  NATO  forces,  and  to  execute  the 
plan  within  76  hours.  For  short-term  planning  the  task 
was  to  formulate  a  group  assessment  of  the  present 
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tactical  situation,  including  a  tentative  identification  of 
surface  contacts,  and  to  (re)task  available  assets  and 
units  in  order  to  accomplish  the  task.  For  near-real-time 
decision  making,  the  Staff  had  to  assess  the  development 
of  the  tactical  situation,  to  reconsider  eventually  the 
identity  of  the  surface  contacts,  and  to  decide  on 
manoeuvring  the  formation  or  to  engage.  Information 
needed  for  planning  and  decision  making,  such  as 
mission,  rules  of  engagement,  observations, 
meteorology,  intelligence,  possible  threats,  etc.,  was 
made  available  electronically. 

2.4  Procedure 

One  week  in  advance,  the  information  used  in  the 
scenario,  consisting  of  30  pages  of  text,  sea  charts,  etc., 
was  provided  to  the  Task  Group  Staff.  Data  collection 
took  place  in  one  afternoon,  from  12  to  6  pm.  After  a 
short  explanation  of  the  aim  of  the  study,  the  subjects 
filled  out  questionnaire  A.  After  lunch,  the  SmartStaff 
concept  was  introduced  and  explained.  Next,  the  subject 
were  trained  for  one  hour  in  using  the  support  tools, 
working  through  a  number  of  exercises  about  forming 
and  presenting  ideas,  and  sending  and  fetching  them. 
The  game  started  at  2.30  and  lasted  for  two  hours.  In  the 
first  hour  of  the  game,  long-term  planning  took  place, 
without  the  Task  Group  Commander  (TGC).  The  plan 
was  subsequently  briefed  to  the  TGC.  In  the  remaining 
time,  short-term  planning  and  near-real-time  decision 
making  was  carried  out,  as  a  reaction  to  a  developing 
threat.  In  these  tasks,  the  TGC  participated  fully.  After  a 
break,  questionnaires  B  and  C  were  filled  in.  The  session 
concluded  with  the  group  discussion. 

2.5  Results 

The  results  came  from  the  three  questionnaires  filled  in 
by  eight  staff  officers,  the  questionnaire  filled  in  by  one 


observer  (the  second  observer  didn’t  show  up),  and 
remarks  made  during  the  group  discussion.  Reliability 
analysis  showed  that  two  items  in  module  2  (efficient 
use  of  individual  and  shared  time)  had  low  item-total 
correlation;  these  items  were  left  out  of  the  analysis.  The 
reliability  of  the  remaining  questionnaires  was 
reasonable  to  good  (Crombach’s  alpha  .97  for 
questionnaire  A;  .74  for  B;  and  .71  for  C). 

In  table  1  the  results  are  presented  for  the  four  specific 
performance  criteria,  derived  from  the  first  four  modules 
from  the  questionnaires.  A  fifth  overall  performance 
assessment  is  added,  derived  from  all  five  modules.  A 
Wilcoxon  rank  test  was  used.  Each  performance  criterion 
was  tested  by  averaging  the  scores  across  the  questions 
of  a  module.  The  table  shows  the  mean  scores  together 
with  their  standard  deviation  (between  brackets),  and  the 
level  of  significance  (p-values). 

The  Task  Group  Staff  assessed  that  the  SmartStaff  based 
environment  supported  their  decision  making  better  than 
their  current  environment  (p  <  0.05  for  both 

comparisons).  Important  to  remark  is  that  the  subjects 
regularly  noted  they  evaluated  in  questionnaires  B  and  C 
the  potential  of  the  SmartStaff  concept ,  not  the  current 
experimental  implementation,  for  which  is  clear  that 
certain  interaction  mechanisms  and  the  speed  of  data 
exchange  can  be  improved. 

SmartStaff  was  found  to  provide  a  less  ambiguous 
shared  picture  shared  of  the  situation  and  the  plan,  when 
the  two  environment  were  compared  directly  (p  <  0.05). 
No  significant  difference  was  found  for  the  absolute 
assessments  of  the  two  environments. 

The  Staff  also  had  the  opinion  that  with  SmartStaff  their 
time  was  used  more  efficiently.  Again,  this  result  was 
only  found  in  the  direct  comparison  (p  <  0.5).  A 
drawback,  put  forward  by  some  subjects,  was  that 
carrying  out  individual  work  in  the  SmartStaff 
environment  may  distract  one  from  shared  decision 
making. 


Performance  criterion 

Mean  Score  (Standard  deviation) 
ABC 

Cr.ot=.  79  Cr.ot=74  Cr.a=.71 

p-values  of  difference 

A-B  C-‘ equal’ 

overall  performance 
(module  1  to  5;  24 

questions) 

2.81  (0.32) 

3.01  (0.30) 

3.77  (0.25) 

.05 

.01 

shared  situation  picture 
(module  1 ;  3  questions) 

2.95  (0.49) 

3.29  (0.42) 

3.49  (0.32) 

n.s. 

.02 

time  efficiency 
(module  2;  3  questions) 

2.96  (0.49) 

2.92  (0.43) 

3.65  (0.49) 

n.s. 

.02 

idea  communication 
(module  3;  5  question) 

2.66  (0.32) 

3.08  (0.55) 

4.22  (0.41) 

.06 

.01 

product  quality 
(module  4;  1  question) 

3.25  (0.46) 

3.13  (0.35) 

3.33  (0.41) 

n.s. 

n.s. 

Cr.a:  Crombach’s  alpha;  n.s.:  not  significant; 

A:  Current  environment;  values  ranging  from  1-4  (bad,  rather  bad,  rather  good,  good); 

B:  SmartStaff  environment;  values  ranging  from  1-4  (bad,  rather  bad,  rather  good,  good); 

C:  Direct  comparison;  values  ranging  from  1-5  (much  worse,  worse,  equal,  better,  much  better). 


Table  1 :  Overview  of  the  results 
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With  SmartStaff,  the  quality  of  ideas  generated  is  not 
better  compared  to  their  current  environment  (p=  0.35 
for  comparison  afterwards,  and  p=0.11  for  direct 
comparison). 

The  Task  Group  Staff  was  clear  about  the  value  of 
SmartStaff  for  the  communication  of  ideas.  The  subjects 
unanimously  thought  that  the  SmartStaff  based 
environment  is  better  or  much  better  in  this  respect 
compared  to  the  current  environment.  (p=0.06  when 
indirectly  compared;  0.01  for  direct  comparison).  Also, 
in  their  comments  the  subjects  expressed  the  strength  of 
SmartStaff  on  this  aspect. 

The  assessment  of  the  observer  was  in  accordance  with 
the  assessment  of  the  Task  Group  Staff . 

4  GENERAL  DISCUSSION  AND  CONCLUSIONS 

4.1  The  results 

The  results  have  shown,  that  the  SmartStaff  concept  has 
a  large  potential  to  support  the  planning  in  teams.  In 
several  respects,  a  SmartStaff  based  environment 
provides  better  support  for  team  decision  making  than 
traditional  environments: 

•  the  shared  picture  of  the  situation  and  plan  that  staff 
members  have  is  better  and  less  ambiguous, 

•  time  is  managed  more  efficiently, 

•  presenting  and  communicating  ideas  and  plans  runs 
much  better, 

•  general  support  to  team  decision  making  is  better. 

In  one  respect,  SmartStaff  did  not  have  any  effect:  the 
quality  of  the  ideas  and  plans  were  not  influenced. 

The  latter  result  seems  to  be  in  contrast  with  the 
findings  of  Streitz  and  colleagues  (Streitz,  Geissler, 
Haake  and  Hoi,  1994,  Streitz,  Rexroth,  P.  and  Holmer, 
1997).  They  found  an  improved  output  in  terms  of  both 
quality  and  quantity  when  a  group  designed  a  logo 
together  while  supported  by  individual  workstations  and 
shared  interactive  large  screen  displays.  In  that  task, 
however,  the  members  of  the  team  all  had  the  same 
expertise  and  did  not  work  together  on  a  single  logo; 
they  generated  a  number  of  them  and  then  selected  the 
best  one.  So,  the  type  of  task  and  the  homogeneity  of  the 
staff  and  the  logo  designers  are  different  and  a 
comparison  can  not  be  made  easily. 

It  would  be  interesting  to  find  out  why  the  quality  of 
the  final  plan  did  not  improve.  One  explanation  may  be 
that  in  a  planning  task,  in  contrast  to  real-time  decision 
making,  sufficient  time  is  available  to  make  the  plan 
better.  A  difference  in  output  would  only  be  found  when 
there  are  time  constraints.  In  such  a  situation,  more 
efficient  time  management  will  play  a  critical  role. 

Individual  comments  of  the  subjects  and  remarks 
during  the  group  discussion  pointed  out  a  number  of 
additional  aspects.  The  advantage  of  interactive  large 
screen  displays  to  support  idea  presentation  and 
communication  was  particularly  clear  for  long-term 
planning  and  for  briefing.  A  better  and  less  ambiguous 
shared  picture  was  also  found  for  short-term  planning 
and  near-real-time  decision  making.  Another  positively 


assessed  aspect  was  the  possibility  of  reviewing  the 
course  of  the  decision  making  process.  Time 
management  was  found  to  be  more  efficient:  the  subjects 
found  it  easy  to  change  from  individual  tasks  to 
collective  tasks.  It  was  recognised,  however,  that 
individual  tasks  that  demand  much  concentration,  could 
better  be  carried  out  in  isolation. 

4.2  Limitations  of  the  method 

Firm  conclusions  are  limited  by  the  method  used. 

One  methodological  problem  may  be  the  Hawthorne 
effect:  is  the  effect  not  just  a  result  of  running  an 
innovative  system?  This  may  be  avoided  by  asking  the 
staff  to  work  in  new  environment  for  some  time  and 
measure  again.  Unfortunately,  this  is  practically 
unfeasible.  The  results  show,  however,  that  the  subjects 
were  not  positive  on  one  particular  aspect:  quality  and 
quantity  of  the  ideas  did  not  improve.  Also,  during  the 
group  discussion,  it  became  clear  that  there  was  at  least 
some  reluctance  to  accept  the  new  technology.  These 
two  observations  may  indicate  that  the  subjects  may  not 
have  been  influenced  by  the  Hawthorne  effect. 

Using  a  real  team  instead  of  artificial  teams  has  the 
advantage  of  ecological  validity.  We  have  studied  a  team 
in  its  real  working  environment,  with  members  having 
specific  individual  knowledge,  experience,  and  skills 
who  are  used  to  working  with  each  other.  A  drawback  is 
the  practical  consequences.  It  is  difficult  to  carry  out 
tests  on  a  Task  Group  Staff  since  there  is  only  one  such 
team  in  the  RNLN.  Still,  the  impact  of  their  decision 
making  process  is  enormous,  so  research  is  important, 
even  within  these  methodological  constraints. 

4.3  Conclusions 

Our  aim  was  not  to  fine  tuning  a  particular 
implementation  but  to  carry  out  a  conceptual  evaluation 
in  an  early  phase  in  a  systems  engineering  life  cycle. 
Such  an  evaluation  yields  the  functional  requirements  of 
a  system  (see  e.g.  Sage,  1992).  Neerincx,  Van  Doome 
and  Ruijsendaal  (1999)  show  also  that  support  systems 
can  indeed  be  evaluated  in  an  early  phase.  They 
distinguish  a  task  level  and  a  communication  level 
evaluation,  the  former  can  be  carried  out  far  before  the 
system  is  operational.  The  lessons  of  a  conceptual 
evaluation  are  independent  of  the  state  of  technology, 
and  therefore  last  longer.  Moreover,  it  helps  us  to 
understand  how  teams  work  and  how  they  should  be 
supported. 

The  present  findings  raise  a  lot  of  questions.  Further 
work  will  address  why  the  final  product  of  the  staff  did 
not  apparently  improve.  We  would  like  to  know  whether 
this  depends  on  group  characteristics  (a  heterogeneous 
team  used  to  work  together)  or  type  of  task  (no  time 
constraints).  A  second  question,  following  from  the 
first  one,  is  whether  other  types  of  teams  may  profit  from 
SmartStaff  also.  Does  a  management  board  experience 
the  same  level  of  support?  A  third  question  is  how  team 
planning  and  refinement  is  carried  out  at  a  cognitive 
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level.  For  complex  tasks,  a  human  being  has  a  number  of 
mechanisms  available  for  problem  solving 
simplification,  such  as  satisficing  (Simon,  1978). 
Satisficing  is  a  problem  solving  strategy  often  used  by 
designers:  they  are  satisfied  with  a  solution  to  their 
problem  when  it  satisfies  the  constraints,  without  further 
searching  for  a  better  solution.  It  may  be  the  case  that  the 
product  of  planning  and  refinement  can  only  improve 
when  the  planning  problem  or  the  problem  solving 
strategy  is  more  complex.  If  so,  team  planning  needs 
specific  cognitive  support  to  manage  the  complexity. 
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Introduction: 

Command  and  control  operations  became  increasingly 
more  complex  if  not  chaotic  under  real  battle 
conditions.  At  the  same  time  the  importance  of 
psychosocial  factors  in  battle  management  operations 
increases.  Thus  the  uncertainty  of  decision  making  is 
enhanced.  The  aim  of  our  research  is  to  augment  the 
cognitive  utility  in  command  and  control  management 
processes,  using  new  information  technologies  of 
dynamic  sociometry  for  the  analysis  of  systems,  based 
on  uncertain,  ambiguous,  and  poorly  defined  elements. 

Rationale: 

Under  NO  PEACE,  NO  WAR  conditions 
of  contemporary  conflicts,  targeting  is  a  substantial 
part  of  the  process  of  decision  making.  Decision 
about  targets  for  action  has  been  performed  nearly 
exclusively  by  intuition.  This  is  still  more  valid  in 
decisions  including  the  aspects  of  psychosocial 
operations.  Suitable  objective  methods  of  the 
analysis  of  target  aiming  are  lacking. 

Description  of  methods  employed: 

Bahbouh  in  our  laboratory  developed  a 
complex  method  of  dynamic  sociometry  (or 
graphy)  based  on  fuzzy  logic  and  graph  theory.  The 
method  is  a  qualitative  development  of  the  classical 
Moreno’s  sociometry.  In  general,  it  can  be  used  to 
the  analysis  of  any  system.  We  use  it  to  analyses  of 
social  complexes  (micro-  to  macrogroups)  by 
evaluation  of  intra-  or  intergroup  relations.  The 
result  is  an  objective  description  of  subjective 
interpretation  of  relations  within  a  complex  system. 

It  is  possible,  using  fuzzy  sets,  to  compute 
degrees  of  appurtenance  of  qualitative  data,  and  to 
express  in  this  way  the  relations  of  elements 
(subjects,  individuals,  and  groups)  in  a  complex 
system.  The  final  outcome  of  the  analysis  is 
presented  in  form  of  a  map,  where  the  distances 
represent  social  relations  (near  -  positive, 
sympathetic,  or  distant  -  negative,  aversive)  and 
altitudes  correspond  to  high  or  low  social  positions 
of  elements  in  a  system.  The  highest  position 


usually  in  the  center  of  the  group  is  that  of  the 
“STAR”,  the  lowest,  on  its  periphery,  are  those  of 
“OUTSIDERS”.  An  important  position  is  that  of  the 
“BRIDGE”.  This  is  a  subject  with  a  rather  mediocre 
position,  but  with  prevailing  positive  relations.  A 
bridge  subject  can  play  an  important  social  role  as  a 
mediator  among  different  elements  of  the  complex, 
suitable  to  arrange  coalitions  and  cooperation.  To 
be  more  users  friendly,  the  map  is  completed  by 
isohypses.  The  configuration  of  the  “social  terrain” 
corresponds  to  the  relations  under  analysis,  the 
“slopes”  or  “valleys”  representing  obstacles, 
difficulties  in  mutual  understanding  and/or 
sentiments. 

The  reliability  and  validity  of  the  method 
were  tested  under  real  conditions  of  the  activities  of 
our  Center  as  the  expert  laboratory  of  the  Chief  of 
General  Staff  of  the  Czech  Army. 

Results  obtained: 

Reliability  and  validity  of  the  method  were 
tested  with  success  in  a  group  of  three  subjects 
during  an  experimental  simulated  space  orbital 
mission  (changes  during  153  days  of  social 
deprivation). 

The  method  was  verified  on  a  complex 
group  of  groups  in  the  whole  Czech  Air  Force. 

Preliminary  results  of  these  two  mentioned 
real  situations  were  referred  elsewhere. 

This  year  we  analyzed  relations  during  the 
Kosovo  conflict  on  the  level  of  macrosocial 
processes  by  the  aid  of  a  dynamic  sociometric 
model. 

Expeditionary  units  of  Army  of  Czech 
Republic  operate  in  the  Bosnia  and  Kosovo  regions 
as  a  part  of  the  NATO  forces.  Taking  the  post¬ 
hostilities  period  problems  into  account,  the 
knowledge  about  psychosocial  conditions,  local  or 
regional,  is  much  needed.  At  the  same  time  it  is 
important  to  lay  down  interpretation  of  a  possible 
future  development  of  the  strategic  situation  in  its 
broadest  aspects. 
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Social  relations  of  all  macro-groups 
participating  in  the  conflict  were  evaluated  using 
heuristic  approach  and  formed  into  a  sociometric 
matrix.  On  its  base  the  sociomap  of  their  relations 
was  constructed  (Fig.  1).  As  seen  from  the  map, 
NATO,  Russians  and  Milosevic  assume  the  most 
important  social  positions. 

The  prognostic  value  of  dynamic 
sociometry  becomes  apparent  if  used  as  a  model 
(Fig.  2).  When  the  influences  of  NATO,  Russians 
and  Milosevic  were  eliminated  under  simulated 
conditions,  the  whole  social  complex  of  the  Kosovo 
conflict  participants,  representing  a  relatively 
homogenous  situation  as  yet,  disintegrated  into 
three  separate  groups:  two  extremes  (the  Serb 
grouping  and  the  Albanian  one)  and  one  central 
(Montenegrins). 

Relatively  deep  “valleys”,  separating  the 
inimical  parties,  witness  deep  discrepancies. 
Without  doubt,  if  the  external  pressures  of  NATO, 
Russians  and  Milosevic  would  be  absent,  an 
immediate  blowing  up  of  hostilities  and  violence 
would  result. 

An  important  problem  is  the  verification  of 
similar  social  models.  This  can  be  accomplished  in 
two  ways: 

(1)  Taking  intuitively  into  account  some 

logic  of  events. 

(2)  In  comparison  with  real  evolution.  In 

this  relation  I  should  like  to  emphasize  that 

this  prognosis  was  accomplished  on  Oct. 

31st  1999. 

Conclusions: 

Dynamic  sociometry  makes  a  deep 
analysis  of  internal  and  external  relations  of  any 
system  possible,  pointing  towards  weak  spots  in 
system‘s  relations,  asking  in  social  systems  for  the 
leading  “Star”  or  neglected  “Outsider”  within  the 
system,  respective  for  the  “Bridge”  only.  It  is 
applicable  to  any  intricate,  uncertain  and 
ambiguous  complex  system  whatsoever. 

Understanding  human  relations  using 
sociomapping  proved  in  practice  as  a  potent 
prognostic  aid.  We  emphasize  that  the  use  of 
dynamic  sociometry  is  not  limited  to  psychosocial 
events  only.  The  sociomap  is  a  reliable  and  valid 
model  of  complex  situations.  It  is  possible  to 
demonstrate  the  expected  future  course  of  events 
resulting  from  command  decision  making  under 
extremal,  poorly  defined  and  uncertain  situations. 


Legend  Figure  1  and  2 

FIG.  1:  Sociomap  of  the  situation  in  Kosovo, 
October  1999 

Legend: 

NATO -NATO  forces 
RUS  -  Russian  Units 
Mil  -  Milosevic  Party 

Alb  -  Albanian  Republic 
Rug  -  Rugovoi  Party 
UCK  -  Kosovo  Liberation  Army 
Bos  -  Boshniaks 

MD  -  Montenegro  Democrats 
MAC  -  Macedonian  Republic 
SD  -  Serb  Democratic  PArties 

MAD  -  Montenegro  Antidemocratic  Parties 
SAD  -  Serb  Antidemocratic  Parties 
SARM  -  Serbian  Army 


FIG.  2.:  Simulated  situation  in  Kosovo  October 
1999, 

after  exclusion  of  Milosevic,  Russia  and  NATO 
forces. 
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Fig.  1:  Sociomap  of  the  situation  in  Kosovo,  October  1999  (see  legend) 


Fig.  2:  Simulated  situation  in  Kosovo  October  1999, 
after  exclusion  of  Milosevic,  Russia  and  NATO  forces. 
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Introduction 

The  United  States  (US)  Department  of  Defense  initiated 
a  program  in  1997  called  the  Military  Operations  in 
Urban  Terrain  Advanced  Concept  Technology 
Demonstration  (MOUT  ACTD).  MOUT  ACTD  is  a 
joint  US  Army-Marine  Corps,  program  led  by  the  US 
Army  Soldier  and  Biological  Chemical  Command.  The 
MOUT  ACTD’s  charter  is  to  seek  technologies  that 
satisfy  32  jointly  derived  requirements  specific  to 
operations  in  ‘built  up’  or  urban  areas.  MOUT  ACTD 
evaluates  these  candidate  technologies  for  military  utility 
and  transitions  the  successful  candidates  to  acquisition 
programs  for  further  development  and  fielding. 

One  of  the  determinants  of  military  utility  that  the 
MOUT  ACTD  program  uses  is  situation  awareness  (SA) 
-  specifically,  the  influence  of  SA  on  individual  and 
force  effectiveness  as  a  result  of  the  use  of  MOUT- 
related  technologies.  SA  is  defined  here  as  the  warrior’s 
ability  to  quickly  perceive  and  then  discriminate  between 
facets  of  the  tactical  environment,  to  accurately  assess 
and  reassess  the  where,  when  and  why  of  that 
environment,  to  then  know  and  understand  the  nature  of 
the  tactical  situation  and  to  extrapolate  near  term  courses 
of  action  based  on  this  understanding.  This  paper 
describes  the  process  by  which  the  MOUT  ACTD 
program  developed  and  implemented  a  method  for 
determining  the  impact  of  SA  on  individual  and  force 
effectiveness. 

Background 

Behavioral  scientists  have  investigated  SA  to  a  great 
degree  in  U.S.  Navy,  Air  Force,  and  Army  aviation 
communities.  SA,  as  it  relates  to  dismounted  infantry 
operations,  is  an  emerging  area  of  study.  The  excellent 
research  in  this  field  conducted  by  the  aviation 
community,  in  both  individual  and  crew  SA,  has  made  it 
possible  for  behavioral  scientists  to  formulate  new 


approaches  of  investigation  of  SA  for  the  dismounted 
war  fighter  and  small  infantry  unit. 

With  the  advent  of  advanced  technologies  and  battlefield 
digitization,  materiel  developers  are  now  required  to 
provide  connectivity  between  the  dismounted 
infantryman  (DI)  and  the  digitized  battlefield  through 
novel  communications,  sensing  and  command,  and 
control-enhancing  technologies.  For  example,  many  U.S. 
soldiers  may  soon  be  provided  wearable  computers  to  aid 
individual  and  small  unit  command  and  control 
operations.  The  technologies  incorporated  in  these 
computers  include  advanced  sensors,  communications, 
and  navigation.  These  technologies  have  the  potential  to 
directly  impact,  both  positively  and  negatively,  the 
ability  of  the  war  fighter  to  perceive  his  or  her 
environment  and  to  understand  his  or  her  place  in  it.  The 
war  fighter  must  be  able  to  interface  with  these 
technologies  to  best  employ  the  capabilities  they  provide. 
Research  needs  to  be  done  to  first  understand  the  impact 
these  capabilities  have  on  the  warfighter’s  cognitive 
abilities  and  ultimately  to  interweave  the  output  of  these 
capability-enhancing  technologies  into  the  infantryman’s 
decision-making  processes  that  ultimately  impact 
battlefield  outcomes. 

SA  Measures  of  Effectiveness  and  Performance 

There  is  a  variety  of  factors  that  indicate  how  effectively 
a  war  fighter  operates  in  a  battlefield  environment.  How 
individual  and  small  units  employ  new  technologies  is  a 
function  of  this  effectiveness.  The  operational 
effectiveness  factors  include  casualty  ratios,  logistics 
resupply,  combat  support,  lethality,  and  survivability. 
Another  of  these  factors  is  SA. 

Mission  accomplishment  is  the  critical  operational  factor 
in  determining  the  relative  military  utility  of  a  piece  of 
technology.  The  basic  premise  of  the  MOUT  ACTD  SA 
effort  is  that  the  more  situationally  aware  a  force  is,  the 
more  lethal,  mobile,  and  survivable  a  force  is. 


Paper  presented  at  the  RTO  HFM  Symposium  on  “Usability  of  Information  in  Battle  Management 
Operations”,  held  in  Oslo,  Norway ,  10-13  April  2000,  and  published  in  RTO  MP-57 . 


8-2 


Quantifying  this  was  and  is  the  ultimate  goal  of  the 
MOUT  ACTD  SA  effort. 

In  order  to  reach  the  goal  of  developing  a  methodology 
for  quantifying  the  influence  of  SA  on  performance,  it 
was  necessary  to  find  a  way  to  determine  its  value  in  a 
field  environment.  The  challenge  was  to  define 
objective,  field  expedient,  operationally  based  measures 
for  determining  the  influence  SA  has  on  mission 
performance.  To  accomplish  this,  a  panel  of  experts  was 
convened  at  the  US  Army  Research  Institute,  Ft. 
Benning  Georgia,  USA.  The  charter  of  this  panel  was  to 
determine  measures  for  SA  that  would  quantify  changes 
in  combat  effectiveness.  The  panel  was  composed  of 
active  duty  and  retired  Army  and  Marine  Corp.  officers 
and  enlisted  personnel,  training  experts  and  behavioral 
scientists.  The  panel  met  on  four  separate  occasions 
corresponding  to  scheduled  experiments  during  which 
particular  SA-impacting  technologies  would  be  used. 

Goal-Directed  Knowledge  Elicitation  Technique 
A  formal  process  was  needed  to  solicit  the  required  field- 
expedient  measures.  The  panel  used  what  came  to  be 
called  the  Goal-Directed  Knowledge  Elicitation 
Technique  (GDKET)  to  accomplish  this.  This  technique 
was  developed  as  a  way  to  solicit  situation-specific 
mission  needs  from  subject  matter  experts  (SMEs). 
These  mission  needs  were  established  to  provide  the 
means  by  which  specific  mission  goals  were  reached  and 
the  knowledge  of  what  must  occur  on  the  battlefield  in 
order  to  reach  those  goals,  which  served  as  the 
foundation  of  the  SA  measures. 

GDKET  was  developed  based  on  the  goal-directed  task 
analysis  described  by  Endsley  in  “Situation  Awareness 
Information  Requirements  for  En  Route  Air  Traffic 
Control”  (Endsley  &  Rogers,  1994).  The  general 
approach  described  by  Endsley  and  Rogers  is  the  same 
used  here;  however,  the  specifics  of  expert  knowledge 
elicitation  and  requirements  analysis  differs  in  that 
GDKET  uses  a  panel  of  experts  to  identify  mission 
goals.  GDKET  also  relies  on  the  interplay  of  the  experts 
during  role-playing  to  generate  requirements  that  form 
the  basis  of  the  SA  measures  of  performance.  The 
behavioral  scientist  serves  as  an  observer  and  facilitator 
to  cull  the  requirements  from  the  panel’s  discussions  in  a 
non-invasive  way.  In  GDKET,  the  experts  assist  in 
analyzing  the  requirements  and  devising  the  measures. 
Emphasis  is  placed  on  obtaining  field  expedient 
measures,  and  the  GDKET  approach  allows  these  to  be 
obtained. 

In  the  GDKET  mission  vignettes  were  developed  that 
incorporated  individual  and  small  unit  tasks  during 
which  MOUT  ACTD  technologies  could  be  employed. 
An  example  of  one  such  mission  vignette  was  clearing  a 
building  of  enemy  troops.  The  panel  discussed  the 
mission  vignette  in  order  to  achieve  a  common 
understanding  of  the  mission.  Experts,  in  turn,  were 
queried  about  their  role  in  the  mission  (e.g.,  platoon 


leader,  company  commander,  squad  leader).  After  this 
had  been  established  the  vignette  was  “role-played,” 
thereby  soliciting  the  dynamic  elements  of  the  mission 
including  tactics,  techniques  and  procedures  (TTPs)  and 
standard  operating  procedures  (SOP).  All  this  was  done 
independently  of  technologies  employed  during  the 
mission. 

The  panel  members  were  queried  about  their  tasks  as 
well  as  their  intent  at  different  levels  of  granularity  with 
respect  to  the  mission.  For  example,  in  the  building 
clearing  vignette,  the  squad  leader  was  asked  to  describe 
his  activities  during  the  mission.  He  described  his 
mission  goal  first  (clear  a  floor  in  the  building  of  enemy 
soldiers)  then  general  mission  tasks  (e.g.,  providing 
status  reports  to  his  platoon  leader).  Based  on  the 
mission  goal,  sub-goals  and  supporting  tasks  were 
identified.  An  example  of  a  sub-goal  was  to  ensure  that 
the  squad  has  adequate  supplies  of  ammunition  and 
water  during  the  mission.  Supporting  tasks  included 
periodic  querying  of  fire-team  leaders  about  their 
ammunition  status  and  passing  that  information  to  the 
platoon  leader. 

A  list  of  operational  requirements  was  generated  and 
these  formed  the  basis  of  the  SA  measures.  The 
following  is  an  example  of  some  of  the  information 
requirements  generated  for  a  squad  leader  during  a 
building  clearing  mission  and  the  resulting  measures: 

Squad  Leader  must  know: 

•  Location  of  platoon  leader 

•  Location  of  other  squads 

•  Ammunition,  water  and  equipment  status 

•  What  rooms  and  floors  have  been  cleared  -  how 
many  are  left 

•  Status  of  fire  teams 

•  Room  and  floor  layouts  -  blueprints 

•  Location  of  the  enemy  -  what  floor 

•  Location  of  non-combatants  and  animals 

•  Casualties 

•  Prisoner  of  war  collection  points 

•  Location  of  Platoon  Sergeant  for  resupply 

•  Some  of  what  the  platoon  leader  knows 

Measures: 

•  Measure  the  information  actually  reported  against 
the  expected  information.  Based  on  TTPs  and  SOPs 
the  assumption  is  that  communications  within  a  unit 
will  be  100%  accurate.  The  reality  is  that  some 
messages  do  not  get  through. 

•  Frequency  of  reporting.  How  often  was  a  report 
presented  and  received  versus  when  it  was  expected, 
based  on  TTPs  and  SOPs. 

•  Accuracy  of  report.  Was  the  report  received 
accurately;  was  it  complete  and  did  it  contain  the 
correct/intended  information.  This  is  a  function  of 
quality  of  information  provided  to/and  presented  by 
the  sender 
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•  Timeliness  of  report.  Were  the  reports  sent  when 
required?  This  is  based  on  TTPs  and  SOPs. 

•  Two-way  communication.  Quality  and  quantity  of 
interaction. 

•  Reduced  risk  of  fratricide. 

-Unit-to-unit  proximity 
-Status  and  location  of  friendlies 
-Number  of  wounded 

•  Number  of  correct  decisions  made.  A  function  of 
advancing  the  mission,  momentum  -  are  the  correct 
decisions  made  at  the  appropriate  time? 

•  Speed  and  reliability  of  report. 

The  panel  realized  that  some  of  the  measures  could  be 
used  near  term  to  determine  combat  effectiveness,  but 
some  (e.g.  reduced  risk  of  fratricide)  would  require  long¬ 
term  experimentation  to  determine  the  real  impact. 

At  the  end  of  the  GDKET  exercise,  the  panel  was  made 
aware  of  the  new  capabilities  that  they  would  have 
available  to  them  to  use  during  the  mission.  For  each 
mission  vignette,  the  experts  discussed  the  impact  of 
these  capabilities  on  their  goal,  sub-goals,  and  supporting 
tasks.  The  objective  was  to  determine  if  there  would  be 
any  impact  on  current  TTPs  or  SOPs  as  a  result  of  having 
these  capabilities.  This  review  did  not  generally  change 
the  nature  of  the  TTP  or  derived  measures. 

SA  Metric  Development 

After  development  of  the  general  measures  of 
performance  and  effectiveness  using  the  GDKET,  the 
MOUT  ACTD  program  commissioned  the  U.S.  Army 
Research  Laboratory’s  Human  Research  and  Engineering 
Directorate  (HRED)  Fort  Benning  Field  Element  to 
develop  and  validate  an  SA  assessment  metric  for  use  in 
a  MOUT  ACTD  field  exercise  as  an  objective  measure 
of  SA.  The  purpose  of  this  measure  was  to  evaluate  the 
effect  of  the  MOUT  ACTD  technology  on  the  unit's  SA. 
A  unit  equipped  with  current  technology  was  used  as  the 
baseline  in  this  assessment.  Before  development  of  the 
metric,  several  decisions  had  to  be  made.  First,  the  type 
of  assessment  techniques  to  be  used  had  to  be  selected. 
Second,  the  decision  had  to  be  made  concerning  the  time 
of  administration  of  the  technique.  Third,  the  type  of 
exercise  had  to  be  decided. 

Questionnaire  Assessment  of  Knowledge  Technique 
Endsley  (1995b)  proposed  that  the  ability  to  objectively 
measure  SA  is  critical  for  progress  and  understanding  in 
the  field.  She  critiqued  several  measurement  techniques 
that  have  been  performed  in  the  past  to  objectively 
measure  SA.  These  include  physiological  techniques 
such  as  electroencephalographic  measurements; 
performance  measures  used  to  infer  SA  (e.g.,  time  to 
complete  a  scenario,  loss  exchange  ratio,  etc.);  global 
measures  of  overall  operator  performance  which  give  the 
end  result  of  a  long  string  of  cognitive  processes; 
subjective  techniques  such  as  self-rating  and  observer 
rating;  and  questionnaires  about  SA  knowledge  that 


evaluate  against  reality.  She  suggested  that  the 
questionnaire  method  provides  an  objective  and  direct 
assessment  of  SA. 

Freeze-frame  Technique 

The  questionnaire  method  can  be  administered  during 
several  different  points  in  an  exercise.  It  can  be 
administered  at  the  end  of  an  exercise,  during  the 
conduct  of  an  operator’s  simulated  tasks,  or  using  a 
freeze-frame  technique.  Endsley  (1995b)  found  the 
freeze-frame  technique  to  be  more  timely  than  the  post 
test  questionnaire  and  less  disruptive  than  the  on-line 
questionnaire.  However,  other  authors  have  found  fault 
with  the  freeze-frame  technique.  An  often-stated 
criticism  of  the  freeze-frame  technique  is  that  it  is 
intrusive  since  it  induces  a  temporary  halt  in  the  scenario 
(Sarter  &  Woods,  1991).  However,  in  a  simulation  using 
fighter  pilots,  Endsley  (1995b)  found  that  the  freeze- 
frame  technique  did  not  affect  subjects’  performance. 
She  stated  that  the  subjects’  SA  did  not  have  a  chance  to 
decay  during  the  freeze-frame  before  the  SA  simulation 
resumed.  In  other  words,  the  SA  was  still  intact  and  the 
freezing  of  the  scenario  did  not  have  an  adverse  effect  on 
the  outcome  of  the  scenario. 

Free-plav  Exercises 

Realistic  aviation  simulators  have  facilitated  the  study  of 
SA  in  the  aviation  field.  Pilots  can  be  placed  in  the 
cockpit  of  simulators  that  are  almost  indistinguishable 
from  the  real  thing.  The  pilots  can  then  be  presented 
with  stimuli  that  are  carefully  controlled  and  processed. 
The  response  choices  to  these  stimuli  are  few  and  the 
correct  response  choice  is  known  when  the  simulation  is 
developed.  However,  realistic  simulations  of  the  full 
range  of  infantry  activities  that  allow  assessment  of  SA 
have  not  been  developed.  The  infantry  environment  is 
extremely  dynamic  and  interactive,  with  many  possible 
decision  choices.  An  infantry  simulation  exercise  cannot 
be  limited  to  a  single  infantryman,  because  he  operates 
as  part  of  a  unit  or  team  and  against  other  units  or  teams. 
Team  SA  requires  a  much  more  complex  assessment 
than  does  combining  the  assessment  of  SA  of  individual 
team  members.  It  requires  assessment  in  its  own  right 
because  it  involves  unique  activities  such  as  coordination 
and  information  sharing  (Salas,  Prince,  Baker,  & 
Shrestha,  1995).  The  decision  choices  in  such  an 
exercise  are  often  numerous  and  they  result  in  a  decision 
choice  matrix  that  is  very  complex.  Also,  outcomes  are 
numerous  and  cannot  be  predicted  before  the  exercise. 
For  example,  a  scenario  planner  has  no  way  of  knowing 
how  often  an  infantryman  may  shoot  during  an  exercise 
and  thus  does  not  know  ahead  of  time  the  correct  answer 
to  the  SA  question  “how  many  rounds  of  ammunition  do 
you  have  left?”  A  further  complication  of  the  ability  to 
simulate  the  infantryman’s  environment  is  the  fact  that 
the  infantryman  is  “moving,  shooting,  and 
communicating”  while  he  attempts  to  maintain  SA.  He 
is  not  contained  in  an  encapsulated  environment  such  as 
a  cockpit.  Because  of  these  factors,  the  only  current  way 
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to  realistically  replicate  most  of  the  critical  aspects  of  the 
environment  is  in  a  free-play  exercise. 

Free-play  exercises  have  been  used  for  many  years  to 
train  soldiers  in  combat  skills.  Units  start  training  using 
free-play  exercises  at  the  squad  level.  They  conduct 
squad  exercises  to  practice  their  basic  infantry  skills 
(e.g.,  fire  and  maneuver,  navigation,  and  weapon  skills). 
Once  these  skills  are  perfected,  squads  come  together 
under  a  platoon  leader  to  test  their  individual  training 
against  an  opposing  force  in  a  free-play  exercise.  The 
platoons  then  unite  under  a  company  commander  to 
practice.  The  final  evaluation  of  a  commander  is  the 
evaluation  of  his  unit  during  a  force-on-force  free-play 
exercise  against  a  well-trained  opposing  force.  The  use 
of  free  play  presents  the  unit  commanders  with  an  almost 
infinite  number  of  decision  possibilities  and 
demonstrates  the  cause  and  effect  outcome  of  the 
decisions. 

Metric  Development 

Once  the  decisions  were  made  to  use  questionnaire  direct 
knowledge  assessment,  freeze-frame  timing,  and  a  free- 
play  exercise,  the  HRED  Field  Element  began  work  to 
develop  the  specific  SA  metric  that  would  be  used  during 
the  assessment.  Development  of  the  SA  metric  involved 
the  following  essential  steps. 

First,  the  developer  acquired  as  much  knowledge  as 
possible  of  all  facets  of  the  scenarios  planned.  This 
included  knowledge  of  the  terrain  on  which  the  event 
took  place;  knowledge  of  the  operations  order  that  was 
provided  to  the  unit  to  include  objectives,  constraints, 
and  the  time  of  day  the  operation  would  take  place;  and 
knowledge  concerning  the  enemy  opposing  force  such  as 
the  type  of  weapons  they  might  carry  and  the  number  of 
enemy  troops.  The  scenarios  and  threat  used  for  the 
exercise  were  standard  scenarios  developed  by  the  Army 
based  upon  a  typical  mission  and  threat.  Scenarios  that 
contain  lots  of  action  by  either  the  enemy  or  the  friendly 
forces  are  ideal  for  use  in  SA  assessment. 

Once  knowledge  of  the  scenario  was  gained, 
determination  was  made  of  how  many  freeze-frames 
were  needed  and  where  to  place  the  freeze-frames  in  the 
scenario.  Infantry  SA  is  not  static,  but  rather  the  result  of 
ongoing  processes  within  the  unit.  Therefore,  a  single 
measurement  point  was  not  adequate.  SA  assessment 
should  be  made  over  a  series  of  important  events  while 
the  unit  is  performing  tasks  (Salas  et  al.,  1995).  In  a 
large  exercise,  the  timing  of  each  freeze-frame  is  critical. 
Endsley  (1995b)  found  that  subjects  in  her  aviation  study 
were  able  to  provide  information  about  their  SA  about  a 
specific  situation  for  as  long  as  6  minutes.  Therefore,  the 
freeze-frames  were  planned  to  take  place  no  more  than  6 
minutes  after  critical  situations  occurred.  Generally,  the 
freeze-frame  should  provide  as  little  disruption  as 
possible.  It  was  decided  that  the  times  just  before 
naturally  occurring  breaks  in  a  scenario  were  good 


locations  for  placement  if  these  occurred  within  6 
minutes  of  critical  SA  elements. 

Next,  the  developer  identified  critical  SA  elements  that 
were  contained  within  the  specific  event.  This 
knowledge  was  acquired  by  holding  an  SME  conference. 
A  group  of  SMEs  that  were  knowledgeable  in  the  field  of 
infantry  and  had  experienced  situations  similar  to  those 
that  would  take  place  in  the  free-play  exercise  were  best 
able  to  determine  what  the  critical  SA  elements  were  for 
the  specific  exercise.  These  SMEs  proposed  SA 
elements  that  would  be  contained  in  the  scenario  and 
then  prioritized  them  in  terms  of  their  importance  within 
the  scenario.  Because  it  was  desired  that  the  amount  of 
time  during  the  freeze- frame  would  be  kept  to  a 
minimum,  only  critical  information  was  assessed.  While 
many  SA-  critical  elements  are  common  across  different 
types  of  scenarios,  some  are  scenario  dependent. 
Therefore,  each  set  of  SA  queries  was  tailored  to  the 
specific  type  of  scenario  that  was  used.  Not  all  critical 
SA  questions  were  naturally  occurring  parts  of  a 
scenario.  Sarter  and  Woods  (1991)  suggested  that 
complex  scenarios  should  include  embedded  events  to 
elicit  key  situation  assessment  responses.  The  inclusion 
of  several  such  events  in  the  scenario  allowed  multiple 
opportunities  for  assessment.  For  example,  an  enemy 
operations  order  was  left  behind  for  the  friendly  soldiers 
to  find.  The  transfer  of  this  information  to  different 
levels  of  command  was  tracked  through  controlled 
queries  at  key  points  in  the  scenario. 

The  developer  then  determined  the  level  of  questions  to 
present.  Endsley  (1987,  1988,  1995)  defined  SA  as  “the 
perception  of  the  elements  in  the  environment  within  a 
volume  of  time  and  space,  the  comprehension  of  their 
meaning,  and  the  projection  of  their  status  in  the  near 
future.”  Endsley’ s  definition  of  SA  discussed  three 
levels:  perception,  comprehension,  and  projection. 

Perception  (Level  I  SA)  is  the  lowest  level  of  SA 
because  it  involves  only  the  conscious  knowledge  that 
something  is  present  in  the  environment.  Level  2  SA  or 
comprehension  is  the  synthesis  of  disjointed  perceptions 
so  that  understanding  of  the  significance  of  the 
perceptions  is  present.  Projection  (Level  3)  SA  is  the 
ability  to  project  future  courses  of  action  based  upon  the 
understanding  gained  from  Level  2  SA.  The  critical  SA 
elements  identified  for  the  scenario  contained  all  three 
levels  of  SA,  but  the  concentration  was  on  Level  I 
because  more  Level  I  critical  elements  naturally  occur  in 
a  short  time  period. 

Development  of  the  specific  SA  questions  to  be 
administered  to  the  soldiers  was  the  fourth  step.  While 
many  of  these  are  natural  extensions  of  the  critical  SA 
elements  that  were  developed  in  Step  3,  the  explicit 
wording  of  the  questions  is  critical  to  the  assessment 
outcome.  For  example,  when  asking  a  platoon  leader 
how  many  wounded  he  has,  it  is  important  to  specify 
whether  you  mean  in  the  headquarters’  element  or  in  the 
entire  platoon.  It  is  also  informative  to  have  the 


8-5 


respondent  identify  the  source  of  his  or  her  information 
for  each  SA  question.  The  source  question  identifies 
from  whom  (i.e.,  platoon  leader,  squad  leader,  rifleman, 
etc.)  or  what  (i.e.,  visual  acquisition,  auditory 
acquisition,  over  the  radio,  etc.)  the  information  was 
gleaned.  For  example,  if  the  purpose  of  the  SA 
evaluation  is  to  assess  the  contribution  of  a  technology  to 
the  respondent’s  SA,  then  a  source  question  can  assist  in 
the  determination  of  whether  the  technology  facilitated 
an  increase  in  SA  or  whether  it  was  another  factor  or 
another  type  of  technology. 

Once  all  the  questions  were  refined,  the  next  step  was  the 
definition  of  “ground  truth.”  Ground  truth  is  defined  as 
the  actual  or  “true”  battlefield  situation.  It  serves  as  the 
basis  of  comparison  for  what  the  subject  perceives  the 
situation  to  be.  This  is  the  most  difficult  and  one  of  the 
most  critical  aspects  of  the  development  of  a  SA  free- 
play  experiment.  Without  an  accurate  definition  of 
ground  truth,  an  evaluation  of  SA  cannot  be  made.  In  a 
free-play  exercise,  ground  truth  definition  can  come  from 
a  number  of  sources.  If  you  have  an  instrumented 
facility,  ground  truth  can  come  from  position  location 
devices  or  video  cameras.  Even  if  instrumentation  is 
available,  it  is  good  to  have  a  backup  source  of  ground 
truth.  Evaluator  controllers  are  SMEs  who  accompany 
the  unit  members  (friendly  and  enemy)  during  the 
exercise  and  are  an  excellent  source  of  ground  truth. 
Therefore,  for  each  SA  question,  a  matrix  was  developed 
to  illustrate  the  source  for  ground  truth.  Sometimes  more 
than  one  source  was  developed.  For  example,  if  the  SA 
question  asked  a  platoon  leader  how  many  enemy  were 
on  his  objective,  ground  truth  was  obtained  from  video 
cameras  on  the  objective,  from  position  location  devices 
on  the  enemy  troops,  and  from  evaluator  controllers  co¬ 
located  with  the  enemy. 

The  final  step  was  the  SME  ground  truth  questionnaire 
that  was  developed  after  the  SA  questionnaire  via  the 
ground  truth  source  matrix.  A  different  questionnaire 
was  developed  for  the  evaluator  controllers  at  each 
location  to  gather  information  about  what  happened 
during  each  freeze-frame.  For  example,  the 
questionnaire  for  the  evaluator  controller  located  with  the 
opposing  force  addressed  the  questions  in  the  SA 
questionnaire  concerning  the  opposing  forces.  Questions 
covered  such  things  as  how  many  opposing  force  were 
killed  during  the  preceding  frame,  where  the  opposing 
force  was  located  during  the  proceeding  frame,  and 
whether  any  civilians  were  co-located  with  them.  The 
questionnaire  for  the  evaluator  controller  located  with  the 
platoon  leader  asked  questions  concerning  his  location 
during  the  previous  frame  and  any  orders  he  may  have 
given  during  that  time.  The  ground  truth  questions  must 
be  very  specific  and  carefully  worded  just  like  the  SA 
questions,  or  the  responses  will  not  always  be  useful.  At 
least  one  ground  truth  question  should  be  developed  for 
every  SA  question  identified  in  the  ground  truth  matrix 
as  being  addressed  by  the  questionnaire. 


SA  Data  Analysis 

After  the  exercise,  the  data  gathered  from  the  ground 
truth  questionnaires  and  from  the  instrumentation  was 
used  to  develop  an  answer  sheet  to  use  in  scoring  the  SA 
questionnaires.  Once  the  SA  questions  were  scored,  the 
percentage  of  correct  answers  was  computed  for  each 
soldier  (Marshak,  Kuperman,  Ramsey,  &  Wilson,  1987). 
Percentages  were  also  computed  by  level  of  command 
(squad  leader,  platoon  leader,  and  company  commander) 
and  by  battlefield  operating  system  question  categories 
(i.e.,  maneuver,  command  and  control,  mobility  and 
survivability,  intelligence,  combat  service  support,  etc.). 
Chi-square  tests  were  used  to  distinguish  between  the 
baseline  and  the  MOUT  ACTD  technology  SA  levels. 

Validation  of  the  Metric 

Drawing  upon  the  writings  of  Schneider  and  Schmitt 
(1986),  the  military  SMEs  analyzed  the  content  validity 
in  conjunction  with  development  of  the  GDKET 
measures  of  effectiveness  and  with  the  development  of 
the  SA  free-play  metric.  A  job  analysis  was  performed 
by  experts  participating  in  the  evaluation  of  content 
validity  and  domain  sampling  was  used  to  represent  the 
behaviors  or  knowledge  skills  and  abilities  found 
important  for  success.  The  free-play  metric 
demonstrated  both  content  and  face  validity. 

Limitations 

Several  limitations  were  present  during  the  SA 
experiments.  Because  of  the  nature  of  the  free-play 
exercise,  many  uncontrolled  variables  may  have  affected 
the  results  obtained.  These  included  leadership  style, 
bad  weather,  late  nights,  long  days,  and  their  potential 
impact  upon  troop  morale.  Also,  a  free-play  exercise 
involving  teams  (i.e.,  other  squad  members,  other 
squads,  and  other  platoons)  does  not  control  for  the 
effect  of  the  other  individuals’  expertise  and  motivation 
upon  SA.  Last,  but  not  least,  the  exercises  used  during 
the  experiments  lasted  for  long  periods  of  time 
(exceeding  1  hour).  There  was  a  desire  on  the  part  of  the 
experiment  directorate  to  disrupt  the  flow  of  the  battle  as 
little  as  possible.  Therefore,  only  three  freeze-frames 
were  executed  throughout  the  entire  exercise.  This 
resulted  in  many  SA  elements  not  being  assessed. 

Conclusion 

In  this  article  we  sought  to  describe  the  development  and 
application  of  methodologies  used  to  develop 
operationally  based  SA  measures  and  assess  the 
contribution  of  MOUT  ACTD  technologies  to  the  SA  of 
individual  soldiers  and  small  units  during  free-play 
exercises.  The  methodologies  discussed  in  this  paper 
were  successfully  used  to  elicit  field  expedient  measures 
of  SA  and  to  assess  overall  performance  of  individual 
and  small  units  during  experiments.  The  freeze  frame 
methodology,  in  particular,  was  a  valuable  tool  employed 
during  several  MOUT  ACTD  experiments.  It 
demonstrated  the  ability  to  discriminate  between  baseline 
and  technology  conditions  and  the  ability  to  track  a 
learning  curve  over  time.  It  also  demonstrated  both 
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content  and  face  validity.  Because  the  methodology  was 
shown  to  have  merit,  it  will  be  used  in  a  large,  company¬ 
sized,  upcoming  experiment  that  focuses  exclusively  on 
SA.  This  experiment  will  address  TTPs  for  use  primarily 
with  intra-squad  radios  and  the  contribution  of  intra¬ 
squad  radios  to  the  SA  of  the  squad. 
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The  dynamics  of  tactical  missions  are  of  a  specific  nature.  Determined  and  shrewd  exploitation  and  control  of  their 
inherent  real-time,  safety-critical  operational  dynamics  are  vital  for  success  in  a  wartime  or  disaster  scenario.  This  paper 
describes  research  and  development  of  theories,  methods  and  tools  for  modeling,  analysis  and  accident  prevention  in 
precarious  time-critical  air  traffic  control,  process  control,  emergency  response  and  military  operations.  We  performed 
case  studies,  field  studies,  and  experiments  using  a  combined  systems  theory.  Cognitive  Systems  Engineering  and 
psychophysiology  framework.  We  carried  out  Joint  Tactical  Cognitive  Systems  identification,  modeling,  and  synthesis, 
and  investigated  inherent  command,  control,  and  intelligence  activities.  We  found  significant  relations  between 
workload,  time  pressure,  cognitive  complexity,  and  physiological  stress  responses. 


Introduction 

The  Nature  of  Complex  Dynamic  Processes  and 
Operations  can  be  characterized  as  high-risk  activities, 
where  human  and  artificial  team  members  together 
perform  a  task,  which  exacts  extreme  mobility, 
efficiency,  agility  and  endurance.  In  emergency 
management,  air  traffic  control  and  military  operations 
mission  performance  relies  increasingly  on  distributed 
systems  (with  many  team-players,  widely  separated, 
forced  to  co-ordinate  with  one  another)  to  attain  high 
safety  and  effectiveness  without  risking  excessive 
resource  depletion.  Commanders  and  operators  will  in 
the  future  be  executing  missions  with  operational  and 
system  characteristics  that  are  highly  dynamic  and  non¬ 
linear,  i.e.  small  actions  or  decisions  may  have  serious 
and  irreversible  consequences  for  the  mission  as  a 
whole.  In  these  kinds  of  activities  decisions  and  actions 
are  never  isolated  events.  They  occur  in  the  context  of: 

•  Stress  effects. 

•  Uncertain  evidence. 

•  Ambiguous  information. 

•  Time  pressure  and  time  delays. 

•  High  physical  and  mental  workload. 

•  Goal  conflicts  (organizational  and  social  factors). 

•  Minor  actions  that  can  trigger  large  consequences. 

•  Highly  dynamic  and  sometimes  chaotic 
environments. 


Performing  complex,  high-risk,  tactical  operations 
requires  support  by  highly  capable  management.  High- 
capacity  C3I  support  is  needed  to  facilitate 
omnidirectional,  continuous  of  information  flows  from 
the  chief  executive  level  to  the  team-on-site  levels. 
Sometimes  individual  operators  and  sensor  systems  must 
without  delay  be  allowed  to  affect  decisions  and  actions 
of  a  senior  commander.  This  is  beyond  reach  unless 
new,  cutting-edge  solutions  can  support  the  humans  and 
systems  engaged.  The  military  community  calls  for 
ground-breaking  approaches  to  demanding  battle 
management  problems.  Analogous  to  this,  the  art  and 
practice  of  command  and  control,  tactics,  techniques, 
procedures  and  training  are  forced  to  constantly  and 
concurrently  strive  for  perfection.  However,  as  Rochlin 
(1997)  and  others  have  observed,  the  specific  skills  and 
properties  that  systems,  managers  and  operators  have  to 
possess  in  order  to  yield  optimal  mission  performance  in 
such  critical  and  uncertain  situations  are  not  easily 
identified,  and  hence,  they  are  difficult  to  improve. 

Our  underlying  principle  was  integration  of  well- 
established  scientific  disciplines  into  a  pioneering 
research  direction,  Action  Control  Theory ,  a  framework 
specifically  composed  to  facilitate  empirically  based 
conceptual  modeling  of  dynamic,  complex  tactical 
systems  and  processes  and  of  their  states  and  state 
transitions.  The  resulting  models  will  be  used  for 
complex,  multi-level  human-machine  systems  design  in 
the  military,  aviation  and  emergency  response  domains. 


Paper  presented  at  the  RTO  HFM  Symposium  on  “Usability  of  Information  in  Battle  Management 
Operations  ”,  held  in  Oslo ,  Norway,  10-13  April  2000 ,  and  published  in  RTO  MP-57. 
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The  Action  Control  Theory 
Framework 

Action  Control  Theory  (ACT)  is  a  composite  theoretical 
structure,  derived  from  advances  in 

I.  Cognitive  Systems  Engineering  (CSE). 

II.  Systems  Theory,  Control  Theory  and  Cybernetics. 

III.  Decision  Making  in  Complex  Systems  Control  and 
Mission  Command. 

IV.  Psychophysiology. 

The  four  research  constituting  ACT  have  until  now 
developed  along  separate  paths  of  evolution.  However, 
now  it  is  time  to  investigate  what  they  might  offer  when 
implemented  in  an  integrated,  cohesive  and  coordinated 
manner.  Flach  &  Kuperman  (1998)  concluded  that  it  is 
essential  to  develop  a  unified,  proactive,  CSE-based 
approach  in  research  and  systems  design  for  future 
warfare  environments.  We  agree,  and  hold  a  strong 
belief  in  the  power  of  integrative  research  approaches: 

•  Built  on  solid  classical  and  innovative  theoretical 
work. 

•  Using  comprehensive  yet  simple  and  robust 
conceptual  and  specific  models  of  systems,  tasks  and 
missions. 

•  Supported  by  advanced  experimental  and 
measurement  methods,  and  data  analysis  techniques. 

Theoretical  Constituent  I:  Cognitive  Systems 
Engineering 

The  area  of  Cognitive  Systems  Engineering  has  grown 
steadily  since  the  first  significant  contributions  were 
published  in  the  1980s  by  Rasmussen  (1983;  1986),  who 
introduced  the  concept  of  skill-based,  rule-based  and 
knowledge-based  behavior  for  modeling  different  levels 
of  human  performance.  Endsley  (1995)  developed  a 
comprehensive  theory  of  individual  operator, 
commander,  and  team  situation  awareness  in  dynamic 
systems.  Danielsson  &  Ohlsson  (1996)  studied 
information  needs  and  information  quality  in  emergency 
management  decision  making.  This  work  also  applies  to 
the  military  context.  Woods  &  Roth  (1988)  made  a 
comprehensive  review  of  the  CSE  domain.  Hollnagel  & 
Woods  (1983)  made  a  significant  contribution  to  this 
field  by  their  definition  of  a  Cognitive  System  (CS)  as  a 
Man-Machine  System  (MMS)  whose  behavior  is  goal- 
oriented,  based  on  symbol  manipulation  and  uses 
heuristic  knowledge  of  its  surrounding  environment  for 
guidance.  A  cognitive  system  operates  using  knowledge 
about  itself  and  the  environment  to  plan  and  modify  its 
actions  based  on  that  knowledge.  In  complex  systems 
this  is  indisputable.  For  example,  in  Command  and 
Control  (C2)  tasks  in  military  missions  a  multitude  of 
sensor  systems,  communication  systems,  training 
programs,  personnel  and  procedures  are  all  elements  of 
the  total  operational  system.  Viewing  this  system  as  a 
cognitive  system  permits  the  integration  of  all  existing 
control  resources:  operators  and  commanders, 


technological  facilities,  doctrine,  procedures  and 
training  into  a  coordinated  system  that  can  achieve  a 
mission  safely  and  efficiently.  The  use  of  CSE  to  model, 
analyze,  and  describe  such  systems  performing 
hazardous,  real  time,  high-stake  activities  is  a  powerful 
approach,  given  a  sufficient  understanding  by  the 
investigator  of  the  interdependencies  and  linkages 
between  other  research  areas  and  the  CSE  field. 

Theoretical  Constituent  II:  Dynamic  Systems 
Theory,  Control  Theory  and  Cybernetics 

By  the  term  dynamic  system  is  meant  an  object,  driven 
by  external  input  signals  u(t)  for  every  t  and  as  a 
response  produces  a  set  of  output  signals  y(t)  for  every  t. 
From  the  work  of  Ashby  (1956),  Brehmer  (1992)  and 
many  others  it  is  well  known  that  most  complex  systems 
have  real-time ,  dynamic  properties ;  the  system  output  at 
a  given  time  is  not  only  dependent  of  the  input  value  at 
this  specific  time,  but  also  on  earlier  input  values,  and 
that  a  good  regulator  of  a  system  has  to  implement  a 
model  of  the  system  that  is  to  be  controlled.  Put 
otherwise,  Ashby’s  law  of  requisite  variety  (Ashby, 
1956),  states  that  the  variety  of  a  controller  of  a  dynamic 
system  has  to  be  equal  to  or  greater  than  the  variety  of 
the  system  itself. 

An  approach  based  on  control  theoiy  and  dynamic 
systems  can  facilitate  structuring  and  understanding  of 
the  command  and  control  problem.  The  mathematical 
stringency  and  powerful  formalism  of  control  theory 
makes  it  possible  to  describe  and  treat  systems  as 
diverse  as  technical,  organizational,  economic  and 
biological  dynamic  systems  in  basically  the  same 
manner:  as  processes,  or  clusters  of  processes,  with  a 
built-in  adherent  or  assigned  control  system.  The 
concepts  of  control  theory  can  be  used  as  metaphors  in 
research  on  decision  making,  especially  in  multiple- 
player,  dynamic  contexts.  The  notion  that  decision 
making  constitutes  the  regulatory  function  in  command 
and  control  processes  (Orhaug,  1995)  strongly  supports 
the  control  theory  approach.  This  notion  also  supports 
the  fact  that  the  hierarchical  command  structures  of 
military  and  emergency  response  organizations  are 
strongly  coupled  to  both  centralized  and  distributed 
decision  making  principles  (Brehmer,  1988).  Annett 
(1997)  used  control  theory  to  investigate  team  skills. 
This  hints  at  the  use  of  a  control  theory  framework  for 
analysis  and  evaluation  of  command  and  control  in 
tactical  operations.  Four  fundamental  requirements  must 
be  met  (Conant  &  Ashby,  1970,  Glad  &  Ljung,  1989 
and  Brehmer,  1992)  if  control  theory  is  to  be  used  in 
analysis  and  synthesis  of  dynamic  systems: 

1.  There  must  be  a  goal  ( the  goal  condition ). 

2.  It  must  be  possible  to  ascertain  the  state  of  the 
system  {the  observability  condition ). 

3.  It  must  be  possible  to  affect  the  state  of  the  system 
{the  controllability  condition ). 

4.  There  must  be  a  model  of  the  system  {the  model 
condition ). 
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Controlling  Joint  Systems  and  Processes 
The  combined  view  of  control  theory  in  technical  as 
well  in  behavioral  domains  is  crucial  for  success  in  this 
research  area.  When  a  function  is  implemented  at  one 
level  of  abstraction,  represented  at  a  second  level  of 
abstraction  and  controlled  at  a  third  level  of  abstraction 
the  requirement  for  timely  and  complete  information 
varies  accordingly.  On  the  other  hand,  it  is  not  important 
whether  a  function  or  mission  is  carried  out  by  an 
operator  or  by  an  automated  system  under  higher-order 
supervision,  the  operators  and  the  supervisory 
controllers  still  need  to  maintain  an  adequate  situation 
understanding  -  or  situation  awareness. 

If  reliable  and  timely  observation  and  measurement  of 
the  system  output  is  unfeasible,  and  situation 
understanding  cannot  be  based  on  the  information 
supplied  by  the  system,  it  must  be  based  on  the  current 
process  knowledge  and  understanding  of  the  situation. 
Operators  and  controllers  must  compensate  by  means  of 
accurate  system  performance  prediction.  This  prediction 
ability  is  based  on  the  axiom  that  a  cognitive  system 
must  be  able  to  think  ahead  in  time  and  anticipate  the 
dynamics  of  the  process.  To  accomplish  this  a  cognitive 
system  must  solely  rely  on  exact  model  knowledge  of 
the  system  input’s  influence  on  the  system  output.  This 
is  normally  referred  to  as  open-loop  control  Open-loop 
control  can  be  a  cumbersome  and  arduous  task, 
especially  when  the  system  environment  and  the  mission 
context  is  highly  dynamic  and  the  system  process  is 
unstable  and  non-linear,  i.e.  small  changes  or  state 
transitions  in  the  process  can  generate  an 
unproportional,  unpredictable  or  even  chaotic  system 
behavior.  In  some  cases  the  disturbances  can  be 
measured.  It  is  then  possible  to  almost  entirely  eliminate 
the  influence  of  those  disturbances  by  using  feedforward 
control  However,  this  requires  extremely  good  system 
knowledge  of  the  process  that  we  wish  to  control. 
Feedforward  control  is  also  sensitive  to  variability  in  the 
system  dynamics.  The  main  advantage  of  feedforward 
control  is  the  possibility  to  counteract  the  effects  of 
disturbances  before  they  are  visible  as  an  undesired 
deviation  from  the  reference.  Control  theory  has  proven 
that  although  feedforward  control  can  be  considered  the 
perfect  mode  of  control,  it  is  often  only  achievable  for  a 
limited  amount  of  time  due  to  model  error  caused  by, 
among  other  things,  the  time-constants  of  the  process. 
However,  if  the  system  output  can  be  used  to  determine 
the  system  state,  there  is  only  a  limited  need  for  detailed 
knowledge  of  system  dynamics,  and  feedback  control 
can  be  executed.  The  necessary  adjustments  and  updates 
of  the  controller’s  internal  system  model  can  be  made  by 
constantly  measuring  the  deviation  of  the  system  output 
from  the  reference  value.  The  joint  cognitive  system  is 
unstable  without  feedback,  and  thereby  feedback  will  be 
needed  to  correct  deviations  and  compensate  for  the 
incompleteness  and  inadequacy  of  the  internal  system 
model.  Reason  (1997)  emphasized  the  importance  of 
balance  between  feedback  (reactive)  control  and 


feedforward  (proactive)  control.  This  concept  is  crucial 
to  achieve  optimal  C2  performance  in  a  tactical  mission. 
Feedforward  control  is  often  combined  with  feedback 
control  because  of  its  practical  reliability  limitations. 

Theoretical  Constituent  III:  Decision  Making  in 
Complex  Systems  Control  and  Mission  Command 

Brehmer  (1992)  suggested  the  use  of  control  theory  as  a 
framework  for  research  in  distributed '  dynamic  decision 
making .  The  conventional  view  of  decision  making, 
supported  by  normative  theories,  reduces  decision 
making  to  selecting  an  appropriate  action  from  a  closed, 
pre-defined  action  set,  and  to  resolution  of  conflicts  of 
choice.  As  a  consequence,  the  analysis  of  decision  tasks 
focuses  on  the  generation  of  alternatives  and  the 
evaluation  of  these  alternatives  as  in  Multi-Attribute 
Utility  (MAU)  analysis  (Kleindorfer  et  ah,  1993). 
Research  in  dynamic  decision  making  has  been  based  on 
analysis  of  several  applied  scenarios,  e.g.  military 
decision  making,  operator  tasks  in  industrial  processes, 
emergency  management  and  intensive  care  (Brehmer, 
1988;  1992).  Two  things  were  clarified  in  these 
analyses: 

1 .  The  decision  making  was  never  the  primary  task.  It 
was  always  directed  towards  some  goal. 

2.  The  dynamic  character  of  the  assigned  tasks  became 
apparent  in  the  study  of  the  applied  contexts. 

These  results  are  consistent  with  earlier  descriptions  by 
Edwards  (1962),  Rapoport  (1975)  and  Hogarth  (1981) 
of  dynamic  decision  making,  which  Brehmer  (1992) 
summarised  as  follows: 

1.  A  series  of  decisions  is  required  to  reach  the  goal 
To  achieve  and  maintain  control  is  a  continuous 
activity  requiring  many  decisions,  each  of  which 
can  be  understood  only  in  the  context  of  the  other 
decisions. 

2.  The  decisions  are  mutually  dependent .  Later 
decisions  are  constrained  by  earlier  decisions  and, 
in  turn,  constrain  those  that  come  after  them. 

3.  The  state  of  the  decision  problem  changes ,  both 
autonomously  and  as  a  consequence  of  the  decision 
maker’s  actions. 

4.  The  decisions  have  to  be  made  in  real  time.  This 
finding  has  several  significant  implications,  and 
they  are  elaborated  upon  in  the  next  section. 

The  real  time  properties  of  dynamic  decision  making 
cause  special  problems: 

1 .  Decision  makers  are  not  free  to  make  decisions 
when  they  feel  ready  to  do  so.  Instead,  the 
environment  requires  decisions  and  the  decision 
maker,  ready  or  not,  have  to  make  these  decisions 
on  demand.  This  causes  stress  in  dynamic  decision 
making  tasks.  In  order  to  cope  with  this  stress, 
decision  makers  have  to  develop  strategies  for 
control  of  the  assigned  dynamic  tasks  and  for 
keeping  their  own  workload  at  an  acceptable  level. 

2.  Both  the  system  that  is  to  be  controlled  and  the 
procedures  and  resources  the  decision  maker  uses 
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to  control  the  system  have  to  be  seen  and  treated  as 
processes.  Dynamic  decision  making  tasks  can  be 
characterized  as  finding  a  way  to  use  one  process  to 
control  another  process. 

3 .  The  different  time  scales  involved  in  dynamic 
decision  making  tasks  have  to  be  monitored  and 
taken  into  consideration .  In  most  situations  the 
active  agents  in  a  dynamic  system,  such  as  the 
directly  involved  operators  and  their  closest 
commander  or  squad  leader,  operate  in  a  time  scale 
of  seconds  to  minutes.  Their  commanders  and  their 
command  and  control  systems  operate  in  time 
scales  of  hours  to  days. 

An  application  of  this  approach  in  studies  of  distributed 
decision  making  in  dynamic  environments  such  as  fire 
fighting  and  rescue  missions  was  described  by  Brehmer 
&  Svenmarck(1995). 

Naturalistic  Approaches  to  Decision  Making 

Zachary  &  Ryder  (1997)  reviewed  decision  making 
research  during  the  last  decades  and  elaborated  on  the 
recent  major  paradigm  shift  in  decision  theory.  The  shift 
is  from  analytic,  normative  decision  making  procedures 
described  in  Kleindorfer  et  al.  (1993)  to  Naturalistic 
Decision  Making  (NDM),  developed  and  described  by 
Klein  (1993a;  1993b),  Zsambok  &  Klein  (1997)  as  well 
as  by  Klein  &  Woods  (1993).  NDM  applies  to  many 
dynamic  and  potentially  dangerous  areas  of  activity  such 
as  military  missions,  air  traffic  control,  fire  fighting, 
emergency  response  and  medical  care.  The  essentials  of 
this  paradigm  are  condensed  below: 

•  Human  decision  making  should  be  studied  in  its 
natural  context. 

•  The  underlying  task  and  situation  of  a  problem  is 
critical  for  successful  framing. 

•  Actions  and  decisions  are  highly  interrelated. 

•  Experts  apply  their  experience  and  knowledge  non- 
analytically  by  identifying  and  effecting  the  most 
appropriate  action  in  an  intuitive  manner. 

Cannon-Bowers  et  al.  (1996)  reviewed,  commented,  and 
related  the  NDM  approach  to  the  extensive  research  on 
Distributed  and  Dynamic  Decision  Making  described 
above.  They  argued  that  this  was  how  to  overcome  the 
limitations  of  the  notions  of  the  classic  normative 
research  paradigm  in  decision  making.  A  fundamental 
element  of  NDM,  the  Recognition-Primed  Decision 
(RPD)  model,  was  presented  in  detail  in  Klein  (1993a) 
and  was  applied  to  complex  command  and  control 
environments  in  Kaempf  et  al.  (1996). 

Tactical  Team  Decision  Making 

Tactical  decision  making  teams  in  the  modem  warfare 
environment  were  faced  with  situations  characterised  by 
rapidly  unfolding  events,  multiple  plausible  hypotheses, 
high  information  ambiguity,  severe  time  pressure,  and 
serious  consequences  for  errors  (Cannon-Bowers  et  al., 
1995).  There  were  also  cases  when  geographical 


separation  or  other  forms  of  distributed  environments  in 
which  the  teams  operate  impose  additional  difficulties 
Brehmer  (1991).  To  be  able  to  adapt  to  these  situations, 
team  members  must  co-ordinate  their  actions  so  that 
they  can  gather,  process,  integrate,  and  communicate 
information  timely  and  effectively.  This  is  particularly 
true  of  complex  systems  where  it  is  difficult  to  assess 
performance  with  a  single  correct  answer,  or  in 
situations  where  several  individual  decision  makers  who 
must  interact  as  a  team. 

Theoretical  Constituent  IV:  Psychophysiology 
Within  joint  cognitive  systems  performing  complex, 
high-risk  military  and  emergency  response  missions 
there  is  a  fundamental  and  profound  connection  between 
human  operator  physiological  stress  response  and 
discrepancies  between  expectancies  and  experiences. 
The  stress  response  is  an  warning  of  an  homeostatic 
imbalance  occurring  (Levine  and  Ursin,  1991).  This 
implies  that  the  concept  of  model  error  from  control 
theory  once  again  can  be  applied.  The  stress  response  is 
also  mobilizing  physiological  resources  to  improve 
performance,  which  is  regarded  as  a  positive  and 
desirable  warning  response.  The  Cognitive  Activation 
Theory  of  Stress  (CATS)  describes  the  phases  of  the 
stress  response  as  an  alarm  occurring  within  a  complex 
cognitive  system  with  feedback,  feedforward  and  control 
loops,  no  less  but  no  more  complicated  than  any  other  of 
the  body’s  self-regulated  systems  (Eriksen  et  al.,  1999). 
The  time  dimension  of  stress  responses  must  be 
accounted  for  very  carefully. 

Models  Derived  from  Action 
Control  Theory 

Tactical  Joint  Cognitive  Systems 

The  point  of  departure  in  our  ACT-based  systems 
modeling  endeavor  was  the  Tactical  Joint  Cognitive 
System  (TJCS),  as  the  system 

•  To  which  a  mission  is  assigned. 

•  To  which  the  operational  command  of  the  mission  is 
commissioned. 

•  To  which  the  responsibility  for  effecting  the  mission 
is  authorized. 

•  To  which  the  resources  needed  for  performing  the 
mission  are  allocated. 

A  Tactical  Joint  Cognitive  System  is  an  aggregate  of  one 
or  several  instances  of  four  principal  sub-system  classes: 

1 .  Technological  Systems ,  for  example  vehicles, 
intelligence  acquisition  systems,  communication 
systems,  sensor  systems,  life  support  systems, 
including  the  system  operators. 

2.  Command  and  Control  Systems ,  consisting  of  an 
information  exchange  and  command  framework, 
built  up  by  technological  systems  and  decision 
makers. 
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3 .  Support  Systems ,  comprising  staff  functions, 
logistic  functions,  decision  support  functions, 
organizational  structures,  and  other  kinds  of  service 
support. 

4.  Tactical  Teams ,  composed  and  defined  according  to 
(Salas  et  al.,  1992): 

“Two  or  more  people  who  interact,  dynamically, 
interdependent^,  and  adaptively  toward  a  common 
and  valued  goal/objective/mission,  who  have  been 
assigned  specific  roles  or  functions  to  perform,  and 
who  have  a  limited  life-span  of  membership.” 


The  concepts  of  a  Tactical  Joint  Cognitive  System  are 
depicted  in  Figure  1. 


Figure  L  The  Tactical  Joint  Cognitive  System. 

Another  important  aspect  is  how  the  actual  mission 
affects  team  performance.  Serfaty  &  Entin  (1997)  drew 
the  following  conclusions  concerning  the  properties  and 
abilities  of  teams  successfully  performing  tactical, 
hazardous  operations: 

•  The  team  structure  adapts  to  changes  in  the  task 
environment. 

•  The  team  maintains  open  and  flexible 
communication  lines.  This  is  important  in  situations 
where  lower  levels  in  a  command  hierarchy  have 
access  to  critical  information  not  available  to  the 
higher  command  levels. 

•  Team  members  are  extremely  sensitive  to  the 
workload  and  performance  of  other  members  in 
high-tempo  situations. 


Tactical  Action  Control  Models 

We  then  turn  our  attention  to  the  Tactical  Action 
COntrol  Model  (TACOM,  Worm,  2000c),  as  illustrated 
in  Figure  2. 


Figure  2.  The  Tactical  Action  COntrol  Model  (TACOM). 

The  principal  components  of  the  TACOM  are  the 
Mission  Environment,  the  Tactical  Joint  Cognitive 
System,  the  Situation  Assessment  function,  and  the 
Cognitive  Action  Control  function,  derived  primarily 
from  the  work  of  Brehmer  (1988;  1992),  Klein  (1993a; 
1993b)  and  Worm,  1998c. 

Mission  Execution  and  Control  Models 

The  next  step  is  integration  of  these  concepts  into  a 
Mission  Execution  and  Control  Model  (MECOM), 
illustrated  in  Figure  3.  The  MECOM  consists  of  one  or 
several  TACOMs  extended  with  control  theoretic 
components,  to  handle  system  disturbances,  model  error, 
and  to  allow  an  adaptive  and  balanced  mix  of 
feedforward  and  feedback  control. 


Figure  3.  The  Mission  Execution  and  COntrol  Model 
(MECOM).  This  is  a  simplified  version  of  the  full  model  for 
greater  clarity  and  for  editorial  reasons.  The  full  model  is 
depicted  in  Worm  (2000b). 
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Model  Combination  and  Aggregation 

The  last  step  in  the  model  formation  process  is 
combining  and  aggregation  of  several  MECOMs  into 
unilevel  and  multilevel  MECOMs,  respectively,  as 
presented  in  Figure  4. 


Figure  4.  A  simplified  example  of  a  MULTl-level  Mission 
Execution  and  COntrol  Model  (MULTl-MECOM). 

Methods:  The  TRIDENT  project 

In  earlier  publications  (Worm,  1998b;  1999b;  1999c)  we 
have  reported  on  the  progress  of  the  Tactical  Real-time 
Interaction  in  Distributed  EnvironmeNTs  (TRIDENT) 
project,  aimed  at  developing  a  coherent  and 
straightforward  package  of  methods  and  techniques  for 
man-machine  systems  analysis  in  the  setting  of  tactical 
mission  scenarios.  The  components  of  TRIDENT  are: 

•  Using  the  Action  Control  Theory  (ACT) 

Framework  for  conceptual  modeling  of  dynamic, 
complex  tactical  systems  and  processes,  of  their 
states  and  state  transitions. 

•  Identification  of  mission  and  unit  state  variables, 
and  of  action  control  and  decision  making 
mechanisms  for  process  regulation  (Worm,  1998a; 
1998b). 

•  Mission  Efficiency  Analysis  (Worm  et  al.,  1 998; 
Worm,  1999a)  of  fully  manned  and  equipped  units 
executing  full-scale  tactical  missions  in  an  authentic 
environment. 

•  Measuring  information  distribution  and 
communication  effectiveness  (Worm,  1998b). 

•  Measuring  workload  by  means  of  the  NASA  Task 
Load  Index  (Hart  &  Staveland,  1988). 

•  Assessing  team  member  psychosocial  mood  by 
means  of  the  Mood  Adjective  CheckList  (MACL, 
SjOberg  et  al.,  1979). 


•  Assessing  situation  awareness  (Endsley,  1995)  as  a 
function  of  mission-critical  information  complexity 
(Svensson  et  al.,  1993) 

•  Measuring  level  and  mode  of  cognitive,  context- 
dependant  control  of  the  team  members,  and 
identifying  what  decision  strategies  were  utilized  by 
the  team  and  team  members. 

•  Applying  reliability  and  error  analysis  methods  for 
investigating  failure  causes  both  in  retrospect  and 
for  prediction  (Hollnagel,  1998). 

•  Validating  identified  constructs  and  measuring  their 
influence  using  advanced  data  analytic  procedures. 

Numerous  battle  management  and  emergency  response 
studies  have  been  carried  out  in  which  we  used  every 
opportunity  to  test,  refine  and  augment  the  modeling, 
measurement,  data  collection  and  analysis  concepts  of 
TRIDENT.  Implementing  our  ideas  for  tactical  mission 
analysis  in  potentially  dangerous,  stressful  and 
cognitively  complex  environments  showed  to  be  very 
effective. 

Using  the  TRIDENT  concepts  for  analysis  and 
evaluation  on  aggregated  system  levels  has  so  far  been 
very  rewarding,  with  high  acceptance  among  the 
subjects;  trained  and  skilled  professionals  performing 
their  daily  tasks  in  their  accustomed  work  environment. 
However,  we  have  also  experienced  some  critique.  It  is 
occasionally  claimed  that  reliability  and  validity  of 
subjective  workload  ratings  are  insufficient.  For  that 
reason  we  considered  incorporating  a  measure  of 
workload  and  stress  which  is  commonly  accepted  in  the 
scientific  community.  We  considered  hormonal  response 
measures,  inspired  by  the  results  of  Svensson  et  al. 
(1993),  who  studied  workload  and  performance  in 
military  aviation,  Zeier,  (1994)  who  studied  workload 
and  stress  reactions  in  air  traffic  controllers,  and 
Holmboe  et  al.  (1975),  who  studied  military  personnel 
performing  exhausting  battle  training. 

We  designed  a  study  in  order  to  elucidate  to  what  extent 
hormonal  physiological  stress  indications  are  linked  to 
the  rating,  observation  and  data  collection  methods 
normally  used  in  TRIDENT  to  assess  workload  and 
tactical  performance.  The  study  is  described  in  Worm 
(2000a),  and  will  be  further  elaborated  upon  in  a  coming 
doctoral  thesis  by  this  author. 
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Preliminary  Results 

The  main  causes  of  mission  failure  were  information 
interpretation  and  distribution  failures,  due  to: 

•  Slow  organizational  response. 

•  Ambiguous,  missing  or  insufficiently  disseminated, 
communicated  and  presented  information. 

•  Equipment  malfunction,  e.g.  power  failure  or 
projectile/missile  impact. 

•  Personal  factors:  inexperience,  lack  of  team  training 
etc. 

Our  empirical  results  through  the  four-year  project  life 
suggest  three  potentially  significant  mechanisms 
influencing  how  the  team  is  able  to  execute  mission 
control,  which  consequently  also  influences  mission 
efficiency: 

1 .  Time-dependant  filtering  functions  like  defense  and 
coping  mechanisms  according  to  the  cognitive 
Activation  Theory  of  Stress  (Eriksen  et  al.;  1999, 
Levine  &  Ursin,  1991). 

2.  Dependence  on  individual  mission  task 
requirements  (Worm,  2000c). 

3.  Balance  between  feedforward  and  feedback  in 
mission-critical  action  control  (Reason,  1997; 

Worm,  2000b). 

Our  theoretical  achievements  were  a  complicated  and 
arduous  venture,  in  that  we  have  constantly  striven  for 
empirical  evidence.  Nevertheless  we  feel  that  we  are 
approaching  a  scientific  breakthrough.  We  argue  that  the 
ACT  /  TRIDENT  approach  will  facilitate 

1 .  Identifying  limiting  factors  of  a  specific  individual, 
unit,  system,  procedure  or  mission. 

2.  Assessing  the  magnitude  of  influence  of  these 
factors  on  overall  tactical  performance. 

3.  Proposing  measures  to  support,  control  and  improve 
insufficient  capabilities  and  contribute  to  successful 
accomplishment  of  future  missions. 

FUTURE  WORK 

We  have  for  a  number  of  years  struggled  towards 
building  a  foundation  for  analysis  and  evaluation  of 
high-stake,  life-threatening  tactical  missions  in  various 
work  contexts.  Although  earlier  results  indicate  that  we 
have  reached  a  workable,  reliable  and  valid  result,  the 
question  is  still  if  our  findings  are  generally  applicable. 
After  preliminary  analysis  of  the  study  reported  on  in 
this  paper,  we  contend  that  studying  individuals  is  a 
effective,  reliable  and  valid  way  to  probe  the  function 
and  efficiency  of  an  organization,  performing  complex 
tasks  in  an  ever  changing  mission  environment.  We  will 
continue  to  work  with  the  data  collected  in  this  and 
earlier  studies,  and  use  the  results  from  the  scenarios 
analyzed  to  tune  and  adjust  the  theory,  models  and 
methods  in  order  to  obtain  a  coherent  and  cohesive 
framework  for  human-machine  systems  analysis  of 


tactical  mission  settings  and  scenarios.  We  will  also 
develop  computerized  versions  of  the  test  instruments,  if 
possible  with  built-in  tools  for  data  analysis  and 
graphical  presentation,  so  that  researchers  and 
investigators  not  familiar  with  the  background  and  early 
history  of  this  project  can  benefit  in  their  own  work 
from  our  achievements. 
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Summary:  This  paper  presents  some  considerations 
regarding  the  information  overload  in  electronic 
warfare,  focusing  on  land-based  tactical  electronic 
systems.  After  a  quick  review  of  signal  environment 
and  signal  processing  tasks  in  electronic  warfare,  two 
particular  cases  are  presented:  emitter  identification 
and  evaluation  &  reporting.  For  these  two  cases, 
some  solutions  to  reduce  information  overload  are 
considered. 

Signal  Environment  on  the  Battlefield 

The  improvements  in  electronics  and  computer 
technologies  have  made  the  number  of  emitters,  both 
civilian  and  military,  increase  dramatically.  As  a 
consequence,  we  are  facing  now  a  very  dense  signal 
environment.  The  expansion  of  signals  occurs  either 
in  space  (ground,  sea,  air  and  satellite  emitters)  or  in 
frequency  (expansion  to  the  higher  frequency  bands). 
Furthermore,  the  signals  in  the  classical  frequency 
bands  have  become  more  complex  from  the  point  of 
view  of  modulation,  coding  and  other  parameters. 

By  definition,  the  signal  environment  consists  of  all 
the  signals  that  reach  the  antenna  of  a  receiver,  within 
the  frequency  band  covered  by  that  receiver. 
Therefore,  not  only  the  signals  of  interest  will  build 
up  the  signal  environment  on  the  battlefield,  but  also 
the  signals  from  friendly  forces,  neutral  forces  and 
many  signals  from  civilian  systems  operating  in  the 
frequency  band  of  the  receiver. 

All  these  signals  can  be  a  target  for  electronic  warfare 
systems,  which  act  to  reduce  or  prevent  the  hostile 
use  of  electromagnetic  spectrum,  and  ensure  its 
effective  use  by  friendly  forces.  There  are  many 
classification  criteria  used  by  electronic  warfare  to 
divide  the  signals  of  interest  in  different  categories, 
but  I  think  the  most  suitable  for  our  purpose  is  the 
classification  criterion  based  on  the  destination  of  the 
emitter.  Using  this  criterion,  the  most  common  types 
of  signals  are: 

•  radio  and  radio-relay  signals 

•  radar  signals 

•  satellite  signals 

•  mobile  telephone  signals 

•  other  type  of  signals 


Radio  signals  can  be  described  by  a  set  of  parameters 
like:  frequency,  bandwidth,  amplitude,  modulation 
etc.  Special  types  of  radio  signals  like  frequency 
hopping  signals  are  described  by  special  parameters: 
hopping  rate,  hopping  bands,  dwell  time  etc. 

Radio-relay  signals  usually  allow  transmission  of 
information  using  more  than  one  channel,  therefore 
an  additional  parameter  can  be  the  number  of 
channels. 

Radar  signals  are  generally  described  by  a  general  set 
of  parameters:  frequency,  pulse  repetition  frequency, 
pulse  width  and  scan  rate.  Particular  types  of  signals 
(stagger,  jitter,  pulse  and  CW  Doppler)  need  special 
parameters  to  be  defined. 

Satellite  signals  have  become  a  target  for  electronic 
warfare  on  the  tactical  field  since  the  satellite  tactical 
systems  were  extensively  used  during  the  latest 
conflicts.  Up-link  frequency,  down-link  frequency, 
amplitude,  polarization,  type  of  information  -  data, 
fax,  voice  -  can  be  only  a  few  parameters  to  describe 
the  signal. 

Mobile  telephone  signals .  Even  the  signal  from  a 
mobile  telephone  is  weak  and  has  a  limited  range,  it 
complicates  very  much  the  signal  environment, 
because  of  the  great  number  of  cellular  phones  within 
the  area,  and  the  special  type  of  the  signal.  This 
mainly  applies  to  low  intensity  conflicts  or  special 
operations  taking  place  in  populated  areas.  Usually, 
the  mobile  telephones  are  not  a  threat  for  military 
systems,  but  they  need  to  be  dealt  with,  in  order  to 
have  a  accurate  picture  of  the  threat 

Electronic  Warfare  Processing  Tasks 

As  we  have  seen  in  the  previous  section,  electronic 
warfare  systems  have  to  respond  to  a  very  broad  band 
of  threats.  Therefore,  the  processing  of  the 
information  concerning  the  threat  is  necessary,  before 
taking  the  appropriate  decision  or  countermeasure. 

The  diagram  of  the  processing  tasks  in  a  tactical 
electronic  warfare  system  used  for  communication 
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systems  is  presented  in  Figure  1.  Each  task  is 
described  in  Table  1. 


Processing  Task 

Role 

Search 

Continuously  scan  a  designated 
frequency  band  in  order  to  detect 
the  threat 

Identify 

Identify  the  emitter  type  by 
matching  the  parameters  of  the 
threat  to  the  parameters  of  the 
library 

Intercept 

Receive  signals  from  the  emitter 

Monitoring  & 

Recording 

Listening,  carrying  out 

surveillance  on  and/or  recording 
of  a  particular  emitter. 

Direction  finding 

Detection  of  the  bearing  of  the 
emitter 

Localization 

Calculation  of  the  emitter 
position  using  two  bearings  at 
least 

Analysis 

Accomplishes  the  thorough 
analysis  of  the  signals,  mainly 
for  unknown  and  scrambled 
signals 

Sensor/Data 

Fusion 

Correlate  data  from  multi-source 
information 

Evaluation  & 

Reporting 

Evaluates  all  the  searching,  DF 
and  analysis  reports  and  issues 
the  Electronic  Order  of  Battle 

Jamming 

Neutralize  the  assigned  threat 

Table  1  Processing  tasks  and  roles  for  an  EW 
System 


The  systems  accomplishing  these  tasks  are  very 
complex  and  diverse.  A  single  operator  can 
accomplish  one  or  more  tasks.  All  tasks  can  be 
remote  controlled,  and  some  can  be  performed 
automatically.  Search,  direction  finding,  localization, 
recording  can  be  automatically  accomplished. 
However,  for  each  task  the  system  needs  the  input 
from  the  operator  and  outputs  data  to  the  operator. 
Some  tasks,  like  emitter  identification,  analysis, 
evaluation  and  reporting  need  the  intervention  of  an 
operator,  at  least  to  validate  the  results. 

It  is  beyond  the  purpose  of  this  paper  to  present 
different  solutions  for  data  acquisition  and  processing 
in  electronic  warfare  systems.  The  nowadays 
technologies  allow  to  build  hardware  equipment  and 


software  packages  capable  of  reaching  up  to  10 
GHz/s  scanning  speeds  and  instantaneous  frequency 
bandwidth  of  several  MHz  in  COMINT  systems,  and 
several  millions  of  pulse/second  capabilities  in 
ELINT  systems.  These  parameters,  combined  with 
others  (increased  sensitivities,  large  databases  etc) 
allow  EW  systems  to  deal  with  low  probability  of 
interception  signals,  or  other  kind  of  exotic  signals. 

Studies  performed  on  human  factors  showed  that 
human  capabilities  have  not  developed  at  the  same 
speed  as  technologies.  Training  an  electronic  warfare 
operator  is  not  so  easy,  and  training  a  crew  is  even 
more  difficult,  because  of  the  differences  among  the 
crewmembers. 

One  of  the  most  important  missions  of  the  electronic 
warfare  is  to  detect  and  warn  of  threats  -  that  means 
real  time  reaction.  We  can  reason  that  the  most 
critical  tasks  in  electronic  warfare  systems  are  the 
tasks  that  cannot  be  performed  folly  automatically 
using  computers  and  need  fast  reaction  times.  Two  of 
them  are  emitter  identification  and  evaluation  & 
reporting  tasks.  Let’s  analyze  them  one  by  one. 

Emitter  Identification 

There  may  be  hundreds  of  radio  emitters,  dozens  of 
radar  emitters  and  other  kind  of  emitters  on  the 
battlefield.  Each  emitter  can  have  several  working 
modes.  If  a  piece  of  equipment  tried  to  deal  with 
everything  on  the  battlefield  without  prior 
preparation,  it  would  be  surely  overloaded  by  the 
considerable  amount  of  data,  no  matter  how 
sophisticated  is  the  electronic  warfare  equipment. 

Therefore,  every  received  signal  has  to  be  compared 
with  the  data  stored  in  the  library.  If  there  is  a  match 
for  enough  parameters,  the  identification  is 
accomplished.  If  not,  the  recorded  signal  is  passed  to 
the  analysis  operator  that  will  have  a  closer  look, 
using  more  precise  tools  and  more  parameters.  These 
operations  are  not  as  simple  as  they  look,  because 
comparison  is  not  always  so  easy  to  make.  There  is 
an  ambiguous  nature  of  the  parameters.  Sometimes,  a 
tracking  radar  in  the  searching  mode  looks  like  a 
navigation  radar  on  the  screen.  For  a  pilot,  that 
difference  can  be  the  difference  between  life  and 
death.  That’s  why  the  operator  has  to  validate  the 
process,  after  the  computer  performs  the  comparison. 
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The  computer  plays  a  major  role  in  the  signal 
identification  algorithm,  but  computers  and  operators 
have  a  different  approach  to  the  information.  Table  2 
shows  how  they  deal  with  the  information  flow. 


FUNCTION 

COMPUTER 

HUMAN  OP. 

Input 

Millions 
pulses/s 
Thousands  of 

commms 

signals 

Optical  input 

Computer  screen 

Model 

function 

Fixed  format 
commands 

A  flexible  way  of 
thinking  based  on 
experience 

Output 

Binary  data 

millions 

bits/s 

keyboard  (50- 

200car/min) 

mouse  clicks 

(60/min) 

voice 

Drawing 

conclusions 

Very 
specific, 
based  on 

precise 
inputs 

Perception  of  the 
whole  electronic 

environment  (even 
when  there  is  not 
enough  data) 

Table  2  Computer  and  human  behaviour  in  EW 
data  processing 


The  first  and  last  characteristics  are  very  important 
when  we  speak  about  signal  identification.  Humans 
can  accept  incomplete  data  and  compile  it  to  achieve 
the  complete  situation  picture.  Whenever  there  is  a 
slight  difference  between  the  detected  and  stored 
parameters  the  operator  has  to  decide  whether  a 
signal  is  to  be  associated  with  a  type  of  emitter  or 
another,  particularly  when  the  computer  says  there  is 
not  sufficient  data  to  give  a  single  and  acceptable 
solution. 

Another  question  is  how  to  avoid  the  limitation 
imposed  by  the  limited  number  of  signals  that  can  be 
processed  by  the  eye  of  the  human  operator.  In  other 
words,  how  can  an  operator  identify  the  dynamic 
representation  of  a  particular  signal  on  a  very 
cluttered  screen.  The  example  shown  in  the  Figure  2 
represents  a  hypothetical  screenshot  of  the  signal 
environment  on  the  tactical  battlefield,  taken  by  a  fast 
direction  finder  system.  The  receiver  system  allows 
the  interception  of  all  types  of  signals  including  the 
agile  ones.  The  hints  (interceptions)  are  directly 
represented  on  the  screen  for  interpretation  and 


identification.  There  are  about  200  hints  from  about 
50  emitters.  It  is  very  difficult  for  an  operator  to 
make  the  distinction  among  fixed  frequency, 
frequency  hopping,  burst  and  other  types  of  signal. 
Only  a  very  skilled  operator  can  separate  some 
signals  and  identify  the  type  of  emitter  and  there  is  a 
significant  probability  of  error. 

What  can  be  done  to  help  the  operator?  First  of  all, 
the  clutter  represented  by  multi-path,  mirror  and  other 
random  signals  can  be  cleared-out  using  statistic 
criteria.  Secondly,  the  fixed  frequency  signals  can  be 
isolated  using  their  constant  parameters  and 
represented  on  the  screen  (white  colour).  The 
frequency  hopping  signals  are  separated  using  the 
bearing  information  (a  significant  number  of  hints 
with  different  frequencies  are  detected  on  the  same 
bearing  -painted  in  yellow).  The  rest  of  signals  can 
be  bursts  (black)  or  other  types  (mobile  phones 
signals,  radio-relay  signals  or  unknown).  Specific 
missions  require  specific  signal  discrimination 
algorithms,  therefore  the  process  described  above  has 
to  be  very  flexible. 

After  the  signal  is  processed,  different  types  of 
signals  will  be  represented  in  different  ways  on  the 
screen,  as  you  can  see  in  the  Figure  3.  This  will 
make  the  life  of  the  operator  significantly  easier. 
This  way  he  can  concentrate  on  using  his  skills  to 
give  the  solution  in  case  of  inconsistency  of  data.  The 
areas  of  the  screen  designated  by  the  numbers  1  and  2 
may  represent  one  or  two  frequency-hopping  signal 
in  each  case.  In  the  first  case  the  operator  has  to 
decide  if  there  are  signals  from  two  hoppers  close  to 
each  other  or  only  from  one  hopper  with  some 
reflections.  Similarly,  in  case  2,  there  could  be  two 
hoppers  or  a  single  hopper  using  two  sub-bands. 

Evaluation  &  Reporting 

Another  critical  task  in  the  processing  flow  is 
evaluation  &  reporting.  The  information  received 
from  different  component  of  the  integrated  system  is 
correlated  using  also  data  from  other  systems  and 
sources  and  the  resulted  information  is  used  to 
elaborate  reports  for  military  commanders. 
According  to  the  Army  Field  Manual  FM  101-5: 
“ Army  operations  produce  tremendous  volume  of 
information.  Much  of  this  information  is  useful,  but 
not  pertinent ,  to  the  commander,  during  decision 
making .  Commanders  and  staff  who  understand  this 
can  avoid  potential  information  overload  by  using 
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effective  systems  to  accurately  and  rapidly  convey 
information  " 

So,  only  the  critical  information  for  the  decision 
making  is  required  during  the  battle.  As  the  main 
functions  of  electronic  warfare  are  to  detect,  warn  of 
threats  and  self-protect,  a  certain  amount  of  EW 
information  is  critical  for  the  commander  and  staff. 

Electronic  Order  of  Battle  (EOB)  is  a  visualization 
application,  showing  in  real  time  the  emitters  of 
interest,  their  level  of  interest  or  threat,  the  allocation 
of  EW  resources  and  the  tactical  interpretation  of  the 
results.  Other  information  of  interests,  such  as  the 
deployment  of  own  troops,  communication  networks 
can  be  also  represented.  The  tactical  situation  is 
presented  using  geographical  features  as  background. 
Figure  4  presents  an  example  of  a  simple  EOB,  using 
a  digital  map  and  an  embedded  library  of  symbols. 
Using  a  VHF  direction  finder  baseline,  close  to  the 
Forward  Line  of  Own  Troops,  the  command  radio 
network  of  the  enemy  is  identified,  Brigade  HQ  and 
Company  HQs  were  placed  on  the  electronic  map. 

It  is  obvious  that  representing  any  detail  of  the 
tactical  situation  on  an  electronic  map  with  plenty  of 
geographical  details  will  make  the  EOB  difficult  to 
read  and  interpret.  There  is  a  practice  of  using 
existing  software  products  (former  reports,  situation 
displays  etc.)  and  adding  new  facts  and  analysis.  The 
result  is  much  more  overcrowded  picture,  with  only  a 
small  part  of  information  being  useful.  It  is  essential 
that  only  the  up-to-date  information  be  rendered  in 
the  report. 

Another  important  problem  of  EOB  is  the 
appearance  and  options  for  the  geographical  features. 
Sometimes,  you  need  some  geographical  features  to 
be  underlined,  while  others  are  obscuring  the  tactical 
information. 

One  solution  to  the  problem  is  the  Geographical 
Interface  Systems  (GIS),  which  use  databases  to  store 
the  geographical  data.  Digital  Chart  of  the  World, 
(DCW),  Digital  Terrain  Elevation  Data  (DTED),  and 
VMAP  are  only  a  few  standards  used  for  digital  maps 
with  GIS.  Figures  5  and  6  represents  a  simple 
comparison,  showing  the  advantages  of  using  a  GIS 
package  to  generate  EOB  situation  displays. 


The  example  starts  from  the  same  picture, 
representing  a  shape  (ARCVIEW)  file,  in  the  Figure 
5  (left  side).  Because  the  files  were  exported  for  the 
purpose  of  this  presentation,  there  seems  to  be  no 
improvement  in  appearance  for  a  shape  file,  but  in 
fact  it  can  be  loaded  faster  and  require  a  smaller 
amount  of  memory.  As  geographical  information  in 
the  shape  file  is  organized  in  a  database  any  report, 
graph,  diagram  or  table  containing  geographical  and 
tactical  data  is  relatively  easy  to  build.  You  can 
search  for  a  particular  village  on  the  map  using  its 
name,  you  can  find  distances  using  mouse  clicks,  or 
you  can  select  the  peaks  having  a  certain  height  with 
road  access  to  install  your  sensors.  There  are  many 
displaying  possibilities. 

Zooming  in  a  bitmap  EOB  can  be  a  nightmare,  while 
zooming  a  shape  image  preserves  the  clarity  and 
require  less  time.  The  comparison  is  presented  in  the 
two  pictures  from  the  middle. 

The  shape  maps  are  organized  on  layers.  One  can 
activate  only  the  useful  layer  (only  one  mouse  click 
can  activate  or  hide  a  layer).  The  top  and  bottom  right 
pictures  represent  the  same  basic  situation.  In  the  top 
right  picture  the  EOB  use  only  two  geographical 
layers:  river  and  population.  In  the  bottom  right  side 
of  the  screen  a  new  layer  was  added  -railways.  The 
tactical  data  can  be  also  organized  on  layers.  Labeling 
allows  you  to  put  only  the  relevant  names  on  the 
map,  by  using  mouse  clicks  (in  this  case  only  the 
names  of  towns  and  villages  close  to  the  military 
headquarters). 

Conclusions 

Some  practical  conclusions  can  be  drawn,  based  on 
the  above  considerations: 

•  There  is  an  increasing  use  of  computers  in 
processing  electronic  warfare  information  on  the 
battlefield  in  order  to  surmount  the  problem  of 
increasing  quantity  of  data,  but  this  does  not 
mean  the  operators  will  be  completely  eliminated 
from  the  system.  The  computers  will  support  the 
limited  human  capability  regarding  input  and 
output  data  and  people  will  use  their  flexible 
thinking  to  cover  the  whole  situation.  This  is  the 
way  to  overpass  the  information  overload 

•  Visualization  technology  will  continue  to  be  an 
important  issue  in  electronic  warfare,  because  the 
people  who  is  involved  in  decision  making  is  not 


getting  smaller.  There  are  a  lot  of  situations 
where  people  need  to  assess,  identify,  make 
interpretation  and  decisions.  Visualization  is  one 
of  the  main  ways  to  prevent  information 
overload. 

The  standard  picture,  with  an  operator  in  front  of 
a  receiver,  turning  knobs  and  looking  narrow 
screens  is  no  longer  valid.  The  new  technologies, 
such  as  VXI,  implemented  the  standard  computer 
interface  as  human-machine  interface  in 
electronic  warfare.  This  allows  mission-driven 
configuration  of  the  EW  systems  and  requires 
less  time  to  train  the  operators. 


EW  RADAR  SYSTEM 
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Figure  1  EW  Processing  Tasks 
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Figure  2  Signal  identification  without  classification 
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Figure  3  Signal  identification  after  classification 
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Figure  4  Electronic  Order  of  Battle 
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Figure  5.  Building  EOB  using  shape  files 
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INTRODUCTION 

This  is  a  challenging  time  for  the  Alliance.  What  are  the  true  warfighting  capabilities  of  the  Alliance?  In 
assessing  those  capabilities,  what  metrics  is  required  to  assess  a  unit’s  joint  readiness?  With  increasing 
demands  being  placed  on  NATO  military  forces,  there  is  a  growing  need  for  Senior  Decision-makers  to 
receive  more  timely,  analytical  and  flexible  readiness  assessments.  Therefore,  the  Alliance  must  be  able  to 
assess  and  prioritize  missions,  essential  tasks  and  operational  requirements.  This  paper  examines  the 
challenges  of  developing  and  maintaining  visibility  of  force  readiness  and  capabilities  within  a  multinational 
environment.  Alliance  and  Member  Nation/Service  policies  on  force  readiness  and  capability  vary  widely 
and  often  provide  only  a  limited  snapshot  of  a  unit’s  ability  to  conduct  its  primary  wartime  mission  and  not 
the  actual  military  operations  they  are  undertaking.  The  challenge  for  the  Alliance  is  to  maintain  readiness  to 
support  near-term  force  requirements  as  well  as  the  long-term  requirements  of  preparing  for  future  security 
challenges.  While  technological  advances  occur  rapidly,  changes  to  readiness  policies  and  reporting 
procedures  move  more  slowly.  This  reality  is  more  complex  within  a  multinational  environment  due  to  the 
requirement  to  extract  data  from  existing  heterogeneous,  legacy  Alliance  databases  in  order  to  present  a 
coherent  view  of  Alliance  forces  and  their  respective  headquarters  activities. 

This  paper  outlines  a  technology  strategy  for  visualization  of  readiness  information  and  for  developing  a 
force  readiness  decision  support  system  for  use  in  a  multinational  environment  to  support  the  monitoring  and 
assessment  of  Alliance  forces  with  respect  to: 

•  Unit  reports  (location,  status,  availability,  etc.) 

•  Determining  tactical  and  operational  readiness  (personnel,  equipment,  training  and  supplies,  operations 
and  interoperability) 

•  Availability  of  military/civilian  airlift,  sealift  and  land  transportation  assets 

•  Availability  of  pre-positioned  equipment 

•  Mobilization  capability 

•  Analyzing  operations  or  contingencies  from  a  force  readiness  perspective 

•  "What  if'  analyses  of  force  capabilities 


Paper  presented  at  the  RTO  HFM  Symposium  on  “Usability  of  Information  in  Battle  Management 
Operations  ”,  held  in  Oslo ,  Norway,  10-13  April  2000t  and  published  in  RTO  MP-57. 
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BACKGROUND 


Maintaining  a  sufficient  level  of  readiness  in  today's  dynamic  political,  fiscal,  and  operational  environments 
presents  a  significant  challenge  for  the  Alliance.  NATO  is  currently  in  the  process  of  restructuring  their 
armed  forces  in  an  effort  to  create  smaller,  lighter,  more  mobile  and  more  rapidly  deployable  forces.  At  the 
same  time,  however,  the  Alliance,  is  engaging  in  an  ever-increasing  number  of  large  and  small-scale 
operations.  Accordingly,  there  is  an  even  more  urgent  requirement  for  consistent  and  reconciled  readiness 
data  by  Alliance  commanders  and  staffs  from  a  readiness  visibility  and  information  process  and  system  that 
will  provide: 


•  An  ability  to  monitor,  measure,  analyze,  and  predict  the  readiness  of  assigned  Alliance  forces  (e.g.. 
Rapid  and  Immediate  Reaction  Forces,  Main  Defense  Forces,  Augmentation  Forces). 

•  Access  to  a  timely  and  accurate  readiness  visibility  and  information  management  system  which 
provides  an  automated  fusion  of  readiness  information. 

•  An  articulation  of  the  cost  of  maintaining  a  high  state  of  readiness  within  the  context  of  small, 
European-based  forces  that  are  being  asked  to  assume  ever  more  increasing  and  broad  requirements. 


CURRENT  READINESS  CHALLENGES  FOR  THE  ALLIANCE 

•  There  are  no  standardized  readiness  metrics  nor  a  readiness  reporting  mechanism  within  the  Alliance 
that  provides  the  status  of  each  member  nation's  capability  to  provide  required  personnel,  combat- 
capable  hardware  and  technology,  appropriate  levels  of  maintenance  and  spare  parts  for  that 
hardware,  and  training  to  ensure  forces  can  actually  conduct  assigned  operations. 

•  Readiness  information  is  currently  provided  through  a  patchwork  of  manual  and  electronic  links. 

•  The  Alliance  has  not  implemented  an  automated  system  and  process  to  fuse  tactical,  operational  and 
strategic  readiness  data. 

•  Manual  readiness  analysis  is  time-consuming  and  manpower  intensive. 

•  Determining  readiness  depends  on  using  an  agreed  to  set  of  metrics  against  which  the  forces  are 
measured.  Applying  varying  metrics,  therefore,  yields  varying  results.  In  addition,  readiness 
measurement  is  often  based  on  subjective  assessments. 

•  While  NATO  forces  are  expected  to  become  smaller  and  more  mobile  organizations,  there  have  also 
been  major  reductions  in  the  number  of  forces  held  at  high  states  of  readiness.  Remaining  forces  are 
a  mix  of  both  lower  readiness  units  as  well  as  core  rapid  reaction  forces  with  the  result  that  readiness 
is  now  measured  in  months  and  weeks,  rather  than  hours  and  days. 

•  With  smaller  forces,  a  small  degradation  in  readiness  is  more  significant  than  before.  Accordingly, 
the  percentage  of  "ready"  Alliance  forces  must  be  larger  in  an  overall  smaller  total  force.  In 
addition,  visibility  of  the  readiness  of  "stay-behind"  forces  is  critical  since  it  is  harder  to  monitor. 

•  Estimating  operational  readiness  involves  aggregating  or  “rolling  up”  readiness  data  from 
subordinate  commands  (tactical  level)  to  higher  commands  (operational  and  strategic  levels).  The 
complexity  in  assessing  operational  readiness  comes  in  trying  to  compare  the  aggregated  data  from 
one  service  with  aggregated  data  of  operational  units  from  two  or  more  other  services. 
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•  Developing  and  implementing  an  enhanced  readiness  visibility  and  information  systems  is  of  value 
only  to  the  extent  that  it  is  accompanied  by  appropriate  modifications  to  doctrine,  concepts  of 
operations,  and  policy  and  the  willingness  of  the  Alliance  to  integrate  them  into  the  operational 
environment.  While  the  formation  of  a  new  NATO  Standardization  Organization  is  a  good  start, 
progress  has  been  slow  in  changing  doctrine,  organization,  and  incorporating  technology  to  ensure 
that  NATO  forces  can  serve  as  an  effective  crisis  management  tool. 

•  The  gap  in  modernization  and  overall  force  reductions  impairs  NATO  readiness.  The  Alliance  can 
currently  mobilize  only  a  small  percentage  of  its  overall  combat  potential  on  short  notice— a 
substantial  decline  since  the  end  of  the  Cold  War. 

•  Europe  may  be  falling  behind  the  United  States  in  technological  capabilities  and  must  depend  on  the 
force  capabilities  of  the  United  States,  particularly  in  the  areas  of  strategic  lift,  logistical 
sustainability,  and  the  gathering,  processing,  and  dissemination  of  intelligence. 


DEFINING  AND  MEASURING  ALLIANCE  READINESS 

Definition.  Readiness  is  a  fundamental  aspect  of  an  effective  armed  force  and  can  be  viewed  as  the 
ability  to  rapidly  mobilize,  deploy  and  sustain  trained  forces  in  an  area  of  operations  for  an  extended 
duration.  Discussions  of  readiness  components  generally  include  the  following  six  elements: 

•  Qualified  people 

•  Combat-capable  hardware  and  technology 

•  Appropriate  levels  of  maintenance  and  spare  parts  for  that  hardware 

•  Appropriate  tactics,  techniques  and  procedures  that  support  the  capabilities  represented  by  the 
qualified  personnel  and  combat-capable  hardware 

•  Training  to  ensure  forces  can  actually  conduct  assigned  operations 

•  The  ability  to  deploy  hardware  and  personnel  to  the  fight 


Measurement.  In  order  to  assess  how  ready  the  Alliance's  military  forces  are,  the  following  criteria 
can  be  applied  and  assessments  made  based  on  the  results: 

•  For  each  measured  unit,  compare  the  required  numbers  of  qualified  personnel  against  the 
numbers  actually  on  hand  and  available. 

•  For  each  measured  unit,  determine  whether  adequate  supplies  and  spare  parts  are  on  hand. 

•  For  each  measured  unit,  determine  and  monitor  the  type  and  amount  of  training. 

•  Determine  the  ability  of  the  sustaining  base  and  infrastructure  to  support  either  major  operations 
or  smaller-scale  contingencies  for  extended  periods. 

•  Identify  whether  the  Alliance  has  developed  and  promulgated  the  appropriate  TTP  for 
conducting  military  operations. 

•  Determine  whether  Alliance  forces  move  quickly  to  wherever  they  might  be  needed. 
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•  Determine  the  extent  to  which  bases,  hangars,  maintenance  depots,  fuel  farms,  training  ranges, 
etc.  are  in  an  "up"  status. 


SPAN  OF  ALLIANCE  READINESS 

Alliance  readiness  exists  at  the  tactical,  operational,  and  strategic  levels.  Readiness  visibility  and 

information  systems  must  be  able  to  support  assessment  requirements  at  each  level. 

•  Tactical  level.  The  level  of  preparedness  of  individual  Alliance  units  to  execute  assigned  missions  with 
available  weapon  and  support  systems.  Are  the  smallest  elements  of  the  Alliance  ready  to  fight? 

•  Operational  level.  The  level  of  preparedness  of  senior  commands  and  joint  task  forces  to  integrate  and 
synchronize  ready  combat  and  support  units  to  execute  assigned  missions.  Can  the  Alliance  effectively 
form  larger,  operational-level  fighting  units  from  forces  of  the  member  nations?  Can  these  organizations 
operate  in  coordinated  ways  with  other  operational-level  units? 

•  Strategic  level.  The  level  of  preparedness  to  support  the  Alliance  military  strategy.  Strategic  readiness 
is  determined  by  senior  Alliance  military  and  political  leadership  by  providing  the  means  to  put  the  right 
forces  in  the  right  place  at  the  right  time  to  fight  the  right  conflict.  Strategic  readiness  is  based  on  the 
aggregation  and  synthesis  of  readiness  data  from  the  tactical  and  operational  levels  and  combined  with 
other  data  such  as  infrastructure  analyses  and  industrial  capabilities. 


ALLIANCE  READINESS  VISIBILITY  REQUIREMENTS 

In  order  to  assess  the  readiness  of  an  integrated  package  of  Alliance  forces  effectively,  an  ability  to  access, 
visualize  and  analyze  the  following  information  is  required. 

•  Unit  identification 

•  Unit  capabilities 

•  Unit  sustainability 

•  Unit’s  ability  to  re-deploy  and  reconstitute 

•  Hierarchical  view  of  the  unit  within  its  component/force 

•  Unit  status  (current  and  projected) 

•  Mobilization  of  total  force  package 

•  View  of  major  training  exercises 

•  Task  Force  staff  training 

•  Force  interoperability  during  operations 

•  Deployment  shortfalls 

•  Cost  associated  with  operations  and  training 


ENHANCING  OPERATIONAL  CAPABILITIES  AND  READINESS  ASSESSMENTS  THROUGH 
TECHNOLOGY  INSERTION 

The  following  proposed  technology  approach  would  provide  Alliance  military  analysts  and  decision-makers 
with  an  automated  decision  support  tool  with  a  visualization  capability  into  stove-piped,  heterogeneous 
readiness  and  deployment  databases.  The  approach  must  provide  a  consolidated  picture  of  Alliance 
readiness  data  and  leverage  this  information  to  improve  the  management  of  resources.  It  will  also  provide  a 
single-point  user-pull  access  to  required  data.  The  following  are  maturing  technologies  that  are  directly 
applicable  to  the  technical  challenge  of  developing  and  implementing  an  Alliance  readiness  visibility  system. 


11-5 


•  Joint  Readiness  Automated  Management  System  (JRAMS).  JRAMS  is  a  user-firiendly,  readiness 
assessment  tool  that  accesses,  compiles,  and  displays  information  from  disparate  readiness  and 
deployment  databases.  JRAMS  allows  military  planners  and  readiness  analysts  to  assess  the  current 
availability  and  preparedness  of  any  combination  of  forces,  supplies,  and  equipment.  JRAMS  permits 
the  rapid  display  of  multiple  scenarios  and  allows  the  user  to  quickly  change  from  viewing  one  potential 
course  of  action  (CO A)  to  another.  Users  can  switch  between  a  graphical  (pipes)  or  text  (spreadsheet) 
view  of  the  data.  Through  the  use  of  Composable  Data  Services  (CDS),  JRAMS  is  able  to  access 
readiness  and  deployment  data  which  was  previously  retrieved  and  tabulated  manually  but  now  available 
from  a  single  graphical  user  interface.  JRAMS  is  currently  transitioning  to  a  three-tier  Java  architecture 
that  takes  advantage  of  proven  technologies  such  as  Java,  XML,  and  HTTP.  These  technologies  are 
easily  applicable  to  Web  browsers  since  they're  the  native  protocol  of  the  Internet  and  allow  JRAMS  to 
be  more  flexible  and  easily  tailored  to  meet  the  requirements  of  multiple  domains.  JRAMS  provides 
quick  access  to  essential  information  on  unit  readiness  and  availability.  It  also  allows  planners  to  explore 
different  options  for  identifying  forces  and/or  force  capabilities  for  various  contingencies.  In  the  past, 
planners  and  readiness  analysts  spent  valuable  time  gathering  information  to  support  plan  development. 
Now,  less  time  is  used  looking  for  information  and  more  time  can  be  devoted  to  developing  and 
assessing  plans.  The  system  uses  “point  and  click”  technology  and  intuitive  interfaces  to  help  users 
arrange  and  filter  data,  accessing  unit  resource,  training  and  commitment  data.  The  system  also  has  an 
export  capability  to  a  number  of  commercial  off-the-shelf  office  software  for  preparing  graphic  and  text 
reports  and  briefings.  JRAMS  enhances  the  clarity  of  presenting  unit  readiness  status  information  by 
color  coding  the  displays. 


Figure  1  -  Notional  JRAMS  View. 


TYCOM  Readiness  Management  System  (TRMS).  TRMS  is  a  web-based  readiness  reporting  and 
analysis  program  that  provides  a  fully  integrated  environment  for  online  analytical  processing  of 
readiness  indicators  to  measure  current  Fleet  readiness,  analyze  readiness  trends,  and  to  facilitate  future 
readiness  and  resource  planning.  TRMS  has  been  successful  because  it  provides  accurate  and  timely 
data,  is  user-friendly,  integrates  a  variety  of  different  readiness  data,  provides  a  common  baseline  for 
hardware  and  software,  provides  a  common  baseline  for  business  rules,  and  eliminates  burdensome 
reporting  practices  for  the  Fleet. 
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Figure  2  -  Notional  TRMS  Readiness  GUI 
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•  Electronic  Watch  Board  (EWB).  An  innovative  information  visualization  capability  called  an 

Electronic  Watch  Board  enables  multiple  concurrent  views  of  up  to  16  different  individual  or  combined 
data  sources  in  a  web-based  environment.  Each  of  the  16  cells  can  be  tailored  to  display  a  variety  of 
analytical  graphics  to  facilitate  user  review  and  decision-making;  each  of  the  cells  is  connected  to  live 
data  and  supports  a  capability  to  display  more  detailed  data  to  permit  further  analysis  of  the  information 
displayed.  .  The  technologies  and  languages  that  can  be  leveraged  in  the  Readiness  Watch  Board  include 
object-oriented  analysis  and  design,  Java,  CORBA,  XML,  RMI  and  UML.  Users  can  select  data 
elements  for  display  and  customize  the  graphical  views  in  individual  cells  in  the  Watch  Board.  The  user 
can  create  pie  charts,  bar  charts,  and  three-dimensional  views  to  compare  and  analyze  selected  data  sets. 


Figure  3  -  Notional  EWB  GUI 


GENERAL  APPROACH  AND  BENEFITS 

•  Employ  Composable  Data  Services  (CDS).  CDS  creates  the  data  access  and  communication 

infrastructure  necessary  to  gather  and  assemble  information  from  multiple  disperses  data  sources  residing 
within  legacy  systems.  CDS  uses  advanced  object  technology  to  stay  ahead  of  evolving  technology. 

CDS  transforms  individual  data  storage  repositories  into  an  information  resource.  The  transformation  is 
accomplished  without  the  need  to  create  large  data  warehouses  or  data  marts  to  accumulate  the 
information.  Existing  data  sources  and  new  evolving  data  sources  are  accessed  through  the  same 
common  information  interface.  CDS  exploits  the  rapid  development  and  cost-savings  benefits  of  CDS. 
CDS  will  allow  the  readiness  developers  to  standardize  interfaces  and  specifications  so  that  software 
built  to  international  standards  can  be  used,  evaluated  and  adapted  in  ways  that  make  sense  to  the 
operational  planners  and  analysts.  CDS  uses  commercial-off-the-shelf  tools  such  as  Rationale  Rose  to 
define  user  capabilities,  data  requirements,  and  create  an  object  model  that  represents  the  business 
applications  and  data  flow  within  the  system.  The  Schema  Server  would  provide  the  data  dictionaries 
and  locations  of  the  data  elements  required  by  the  business  applications.  The  Model  Editor  would  enable 
the  system  designer  to  map  data  to  the  services/business  applications,  and  the  Code-Generator  would 
automatically  generate  the  underlying  services  (e.g.,  query,  proxy,  naming,  and  persistence),  as  well  as 
the  client  objects  of  the  business  applications  and  graphical  displays. 


•  Capitalize  on  the  maturity  and  utility  of  XML.  The  NATO  Consultation,  Command  and  Control 
Agency  (NC3A)  is  examining  whether  NATO’s  use  of  XML  could  help  improve  interoperability 
between  heterogeneous  (national  and  NATO)  C3I  systems.  NATO  has  an  ongoing  data  modeling  effort 
by  a  data  administration  organization.  It  also  has  an  emerging  high  level  technical  architecture  in  which 
the  place  and  role  of  XML  could  be  clearly  identified.  XML  is  platform  independent  and  allows 
different  computer  hardware,  software,  databases  and  communications  protocols  to  exchange 
information.  Data  content  is  separated  from  its  presentation  format,  allowing  customized  views  of  data 
tailored  to  support  specific  user  requirements.  The  use  of  XML  will  drastically  improve  the  user's  ability 
to  find,  retrieve,  and  process  and  exchange  tremendous  amounts  of  information  easily  across  system, 
organizational  and  format  boundaries. 
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•  BENEFITS.  The  benefits  of  employing  CDS  and  incorporating  proven  readiness  assessment  and 
visualization  tools  in  this  technical  approach  are: 

The  technology  increases  the  productivity  and  performance  of  the  developers  by  providing  tools 
for  rapidly  integrating  large  amounts  of  information  from  otherwise  incompatible  systems  into  a 
common  framework. 

The  developers  can  focus  on  application  logic  and  not  on  data  access,  plus  a  reduced  amount  of 
time  and  expertise  is  required  to  implement  robust  services  and  business  applications. 

The  time  and  cost  of  development  and  enhancements  are  reduced  because  of  the  rapidity  by 
which  new  data  sources  may  be  integrated  into  the  common  information  framework,  the 
elimination  of  custom  coding,  and  the  avoidance  of  multiple  point  solutions  in  complex 
architectures. 

CDS  promotes  a  consistent  handling  of  security  issues;  standard  coding  requirements,  policies, 
and  practices  can  be  embedded  within  the  code-generation  process  based  on  defined  rules, 
algorithms,  constraints,  etc. 


PROPOSED  TECHNICAL  APPROACH 

•  STEP  1.  Requirements  modeling  and  analysis.  Alliance  readiness  analysts  in  the  field  and  at 
headquarters  capture  and  iterate  requirements.  Modem  COTS  tools  such  as  Gensym’s  G2  tool  kifcould 
be  used  to  capture  the  requirements,  processes  and  relationships;  model  and  assess  them  within  a 
comprehensive  modeling  and  simulation  framework;  and  engage  operational  end-users  directly  in  the 
knowledge  capture  process  because  of  its  simple  and  intuitive  user  interface. 

•  STEP  2.  What-If  Analyses.  Once  requirements  are  captured  in  the  tool  kit,  “what-if’  analyses  can  be 
rapidly  conducted  to  optimize  the  desired  performance  and/or  test  new  concepts  conceived  by  the  end- 
user  prior  to  system  implementation. 

•  STEP  3.  Proof-of-Concept  system  requirements  definition.  This  phase  would  involve  identification 
and  description  of  a  core  set  of  requirements  and  appropriate  databases  to  be  accessed,  including 
location,  format/content,  access  methods,  specific  Information  Exchange  Requirements  (IER),  and 
security  requirements. 

•  STEP  4.  No  Re-engineering.  A  key  aspect  of  the  technical  approach  is  that  there  is  no  requirement  to 
re-engineer  the  databases  or  consolidate  all  readiness-related  data  in  a  data  warehouse  because  the 
application  of  CDS  technology  would  enables  the  readiness  to  have  direct  access  to  existing  databases. 

•  STEP  5.  Rapid  code  generation.  The  principal  value  of  CDS  is  that  it  promotes  the  rapid  code¬ 
generation  of  many  object  servers  (i.e.  Java  RMI,  EJB,  CORBA)  that  can  access  multiple  heterogeneous 
databases/data  sources. 

•  STEP  6.  Proof-of-Concept  system  design  and  development.  Once  the  core  requirements  have  been 
defined  and  agreed  to  among  the  Alliance  Readiness  IPT,  the  design  and  development  phase  would 
commence.  The  design  and  development  approach  is  illustrated  in  Figure  2  and  summarized  in  the 
following  paragraphs: 


STEP  7.  Iterative  definition  of  future  builds/extensions  to  the  core  system.  Readiness  support 
applications  and  data  requirements  can  be  rapidly  incorporated  in  the  object  model  within  Rational  Rose 
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and  the  Schema  Server,  and  the  new  system  containing  both  the  old  and  newly  designed  services  and 
business  applications  can  be  quickly  code-generated.  In  addition,  legacy  databases  don’t  have  to  be 
changed  and  CDS  developed  applications  can  rapidly  adapt  to  changes  in  data  format  and  structure  in 
order  to  accommodate  changes  in  data  requirements. 


SUMMARY.  From  a  readiness  analysis  perspective,  the  Alliance  is  grappling  with  issues  that  are 
significantly  more  complex  than  the  "jointness"  and  "joint  readiness  assessment"  issues  that  the  U.  S. 

Military  has  been  trying  to  address  for  several  years.  It  will  be  a  major  challenge  to  get  19  member  nations 
to  agree  on  standards  for  readiness  reporting  or  whether  to  allow  Alliance  systems  to  access,  aggregate  and 
display  national  data.  Those  issues  must  be  addressed;  however,  in  order  to  provide  Alliance  commanders 
and  their  staffs  with  consistent  and  reconciled  readiness  information  to  be  used  in  support  of  current  and 
emerging  Alliance  military  force  requirements. 

As  noted  in  the  beginning  of  the  paper,  this  is  a  challenging  time  for  the  Alliance.  Demands  are  increasing 
for  use  of  NATO  military  forces  and  Alliance  commanders  and  their  staffs  must  be  able  to  assess  and 
prioritize  missions,  determine  essential  tasks  and  evaluate  operational  requirements. 

•  Maintaining  a  necessary  level  of  readiness  in  today's  ever  changing  political,  fiscal,  and  operational 
environments  will  continue  to  present  a  significant  challenge  for  the  Alliance.  The  situation  is 
complicated  by  the  fact  that  while  NATO  is  in  the  midst  of  restructuring  their  armed  forces  to  create 
smaller,  lighter,  more  mobile  and  more  rapidly  deployable  forces,  the  Alliance  is  engaging  in  an 
ever-increasing  number  of  large  and  small-scale  operations. 

•  An  Alliance  readiness  visibility  and  information  system  should  allow  for  continuous,  real-time 
readiness  reporting  and  analysis  throughout  the  full  range  of  Alliance  operating  environments  from 
austere  deployed  areas  of  operation  using  commercial  telephone  lines  to  standing  headquarters 
locations.  The  fundamental  requirement  is  for  readiness  visibility  to  support  operational  planning, 
deployments  and  exercises. 

•  To  meet  this  requirement,  readiness  and  readiness  related  data  must  be  identified,  accessed, 
aggregated,  analyzed,  comprehended,  transformed  and  delivered.  Additionally,  the  Alliance  must 
place  greater  emphasis  on  defining  required  readiness  levels  with  the  appropriate  level  of  resources 
applied  against  it.  There  is  a  significant  difference  between  the  required  readiness  of  a  Main  Defense 
Force  unit  versus  more  deployable  units  like  the  Immediate  Reaction  and  Rapid  Reaction  Forces. 

•  The  Alliance  must  be  able  to  strike  the  right  balance  between  maintaining  readiness  to  support  near- 
term  requirements  of  responding  to  theater  crises  and  the  long-term  requirements  of  preparing  for 
future  European  security  challenges.  Accordingly,  Alliance  members  contributing  forces  must  meet 
NATO  requirements  for  enhanced  readiness,  mobility,  sustainability,  survivability  and 
interoperability. 


RECOMMENDATIONS 

1 .  Develop  and  implement  a  readiness  visibility  system  to  allow  Alliance  readiness  analysts  and 
decision-makers  to: 

°  Extract,  evaluate  and  display  data  from  Alliance  readiness  and  deployment  databases. 


O 


Embed  C3I  business  rules  for  navigating,  viewing  and  fusing  readiness  and  readiness-related  data. 
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0  Develop  metrics  to  support  the  requirement  to  integrate  and  synchronize  ready  combat  and  combat 
support  forces  to  execute  assigned  missions. 

0  Enable  rapid  presentation  and  analysis  or  readiness  and  force  capability  data  and  reduce  force 
readiness  assessments  from  days  to  hours. 

°  Enable  military  planners  and  readiness  analysts  to  perform  "what  if'  analyses  and  rapidly  view  the 
impact  of  executing  one  or  more  force  deployment  plans. 

°  Leverage  emerging  data  services  technology  to  provide  heterogeneous  database  functionality  and 
allow  the  retrieval  and  display  of  readiness  information  from  multiple  data  sources. 

2.  Establishing  an  Alliance  readiness  and  force  capability  assessment  system  will,  of  necessity,  require 
changes  to  existing: 

•  Policy  (to  implement  a  more  robust  and  detailed  reporting  requirement), 

•  Technology  (to  develop  a  means  to  access  the  likely  stove-piped  data  sources  of  19  member  nations) 

•  Political  will  (to  allow  the  Alliance  to  create  a  data  collection  capability  that  some  Member  nations 
may  feel  is  intrusive). 

•  Determination  of  standardized  readiness  metrics  for  tactical,  operational  and  strategic  levels  of 
readiness 

•  Determination  of  assessment  metrics 

3.  The  Alliance  should  also  consider  developing  a  tool  to  input  data  from  existing  Alliance  and  member 
nation/service  organizations  in  order  to  provide  a  more  consistent  view  of  forces  and  headquarters 
activities.  The  retrieved  data  presentation  should  include  time,  unit,  requirement,  type  of  activity  and 
other  pertinent  categories. 
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La  visualisation  informatique  du  champ  de 
bataille  decoule  de  notre  maniere  de  voit  dans 
notre  tete  les  informations  que  nous  supposons 
utiles.  Habituer  a  reflechir  aux  mouvements 
d’ unites  sur  des  cartes,  notre  conception  des 
besoins  de  visualisation  reste  liee  a  la  capacite  de 
les  representer  dans  le  temps  et  dans  l’espace.  II 
faut  maintenant  se  demander,  avec  les  nouveaux 
conflits  dans  lesquels  sont  engagees  les  nations 
occidentales,  si  cette  fa?on  d’apprehender  la 
representation  du  champ  de  bataille  reste  vraie  ou 
si  elle  ne  doit  pas  evoluer  vers  d’autres  formes  de 
representation. 

1.  Les  evolutions  strategiques  de  la  decennie 

Dans  le  contexte  bipolaire,  nos  forces  armees 
etaient  preparees  a  faire  face  aux  violences  de 
guerre  des  operations  classiques  de  haute 
intensity  tandis  que  certaines  unites  reservees  aux 
interventions  hors  Europe  connaissaient  d’autres 
types  de  violence  lies  aux  conflits  dits  “  de  faible 
intensite 

Depuis  une  dizaine  d’annees,  la  nature  intra- 
etatique  des  conflits  favorise  de  nouvelles  formes 
de  violence  dont  P interpretation  peut  faire  l’objet 
de  di verses  approches. 

Les  enseignements  tires  des  recents  engagements 
montrent  que  la  violence  peut  relever  de  six 
rationales  distinctes.  II  conviendra  de  les 
analyser  en  terme  de  causes,  de  motivations  et 
d’objectifs.  On  peut  distinguer  les  types  de 
violence  suivants  : 

violence  engagee, 
violence  insurrectionnelle, 
violence  institutionnelle, 

•  violence  mafieuse, 
violence  psychotique, 

■  non  violence. 

Par  ailleurs,  a  Pinterieur  de  chaque  categorie  de 
violence,  les  actes  peuvent  etre  classes  par  degres 
d’intensite  croissant. 


-  Au  niveau  le  plus  bas,  la  violence 
psychologique,  sans  emploi  de  violence 
physique,  s'exerce  par  la  menace,  l’exclusion, 
ou  le  mepris ; 

puis  une  violence  sporadique,  expression 
intermittente  du  passage  probable  a  une 
violence  permanente  ; 

une  violence  ciblee,  dirigee  contre  des 
biens  ou  des  personnes  supposees 
representatives  du  groupe  ou  de  l'organisme  a 
combattre ; 

une  violence  aveugle,  frappant  n’importe 
quel  bien  ou  personne,  ayant  pour  but 
d’instaurer  un  etat  de  terreur  dans  L opinion 
publique  et  qui  contraindrait  les  autorites  a 
negocier ; 

une  violence  legitimee  par  une 
reconnaissance  intemationale  ou  au  moins 
nationale ; 

enfin  une  violence  generalisee,  celle 
d’une  guerre  ouverte,  qui  depasse  le  domaine 
de  la  maitrise  de  la  violence. 

Constatons  la  survivance  des  conflits  de 
souverainete.  Ce  type  de  conflits  dans  le  contexte 
strategique  actuel  ne  menace  plus  les  interets 
vitaux  des  pays  occidentaux.  II  reste  cependant 
plausible  dans  le  cadre  d’un  conflit  regional  entre 
des  puissances  moyennes  s’affrontant  pour 
Petablissement  d’une  ere  d’influence  :  guerre  du 
Golfe,  guerre  entre  PIran  et  PIrak,  guerre  des 
Falklands.  Ce  sont  generalement  des  conflits 
qualifies  de  symetriques.  Les  pays  occidentaux  ne 
peuvent  s’en  desinteresser  en  raison  des  risques 
importants  de  destabilisation  d’une  region  du 
globe  et  des  menaces  qu’ils  peuvent  faire  peser 
sur  les  interets  strategiques  occidentaux.  Ils 
utiliseront  alors  leurs  forces  armees  pour  imposer 
aux  antagonistes  le  retour  a  la  paix  ou  le 
reglement  pacifique  des  differends. 

On  peut  aussi  observer  Pimportance  prise  par  les 
conflits  identitaires.  Ce  type  de  conflit  est  souvent 
dissymetrique.  Le  pouvoir  appartenant  a  la  culture 
dominante  dispose  generalement  de  forces 
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constitutes,  police  et  armee,  organisees  a  la 
maniere  occidentale  et  dotees  d’armements  plus 
ou  moins  sophistiques.  La  violence  s’exercera  des 
deux  cotes  de  maniere  differente.  L’un  par  Tabus 
de  la  force  publique,  P  autre  par  des  moyens  et  des 
modes  d’action  adaptes  aux  reseaux  d’opinion,  de 
logistique,  d’armements,  de  relations.  Au 
terrorisme  des  uns  repond  la  terreur  exercee  par 
les  autres,  a  Poccupation  repond  la  guerilla...  II 
arrive  aussi  que  ces  conflits  soient  symetriques. 
De  nombreux  conflits  africains  ont  ete  des  conflits 
quasi-symetriques,  avec  des  moyens  et  des  voies 
assez  semblables,  pourtant  tres  eloignes  de  notre 
vision  occidentale  de  la  guerre. 

Dans  la  plupart  des  cas,  les  forces  occidentales  se 
pose  en  tiers  (s’ inter-pose)  par  rapport  a  des 
belligerants  ou  auteurs  de  violence.  II  s’agira  de 
moderer  le  conflit,  maitriser  la  violence,  mettre  en 
place  les  conditions  necessaires  a  la  resolution 
politique  du  conflit.  Ce  concept  d’ intervention  ne 
doit  pas  etre  ramene  a  celui  d’ operations  de 
maintien  de  la  paix  instauree  par  l’ONU.  Dans  de 
nombreux  cas,  ll  sera  necessaire  de  recourir  a  la 
force  pour  faire  respecter  les  termes  du  mandat, 
mais  celle-ci  devra  toujours  eviter  la  bascule  dans 
la  guerre  avec  un  ou  des  belligerants. 

Ainsi,  le  cadre  particulier  des  operations  de 
maitrise  de  la  violence  se  situe  plutot  dans  les 
conflits  d’identite,  intra-etatique  et  en 

interposition,  c’est-a-dire  sans  adversaire  designe. 
Mais  ce  n’est  qu’une  approche  generate.  Le  cas 
des  operations  de  contre-insurrection,  par 

exemple,  se  situe  dans  le  cadre  d’un  conflit  soit 
identitaire,  soit  ideologique,  intra-etatique,  sur  le 
territoire  national,  dont  les  auteurs  sont  bien  des 
ennemis  identifies  et  qualifies  comme  tels. 
Comprendre  les  conflits  pour  y  apporter  des 
solutions  est  une  necessity  pour  les  nations 
occidentales.  La  mondialisation  les  contraint  a  se 
donner  un  role  nouveau  de  pacificateurs : 
«  empecher  la  guerre  ».  Si,  dans  le  passe,  il  est 
arrive  a  certains  pays  de  jouer  ce  role  aux  marges 
de  leurs  zones  d’interets  strategiques,  jamais  cette 
idee  n’avait  obtenu  un  tel  consensus  entre 
plusieurs  pays.  En  parallele  aux  buts  politiques  et 
aux  objectifs  militaires  des  guerres  inter-etatiques 
toujours  possibles,  doivent  maintenant  etre  penses 
les  buts  politiques  et  les  objectifs  militaires 
recherches  en  intervenant  dans  les  conflits  intra- 
etatiques.  Dans  ce  cas,  les  forces  mandatees  ne 
cherchent  plus  a  s’opposer  militairement  aux 
combattants.  Elies  cherchent  a  eviter  la 
prolongation  de  la  violence  et  a  permettre  la  mise 
en  place  d’une  solution  politique. 


Cette  approche  est  nouvelle  pour  les  militaires. 
Nee  d’une  vision  tactique  de  l’emploi  des  forces, 
propre  au  contexte  de  la  guerre  froide,  leur 
approche  des  conflits  est  restee  tres 
clausewitzienne.  Certaines  notions  theoriques  de 
Clausewitz  demeurent  pertinentes.  La  guerre  reste 
la  continuation  de  la  politique  par  d’ autres 
moyens.  C’est  bien  l’emploi  qu’en  font  les 
belligerants  et  acteurs  de  violence.  La  fin 
politique  recherchee  par  1’ engagement  militaire 
pour  s’opposer  a  la  violence  consiste  toujours  a 
imposer  une  volonte  politique,  mais  dans  le  sens 
de  la  paix  et  non  plus  par  l’aneantissement  des 
belligerants. 

Si,  comme  le  concevait  Clausewitz,  1’engagement 
reste  un  duel,  il  s’agit  desormais  d’un  duel  des 
volontes,  c'est-a-dire  de  ce  qu’Edward  Luttwak 
appelle  "le  duel  des  esprits  directeurs ".  Il 
n’atteint  pas  et  n’autorise  pas  le  duel  des  armes, 
au  moins  au  niveau  strategique. 

2.  Les  consequences  sur  la  representation  de 
Pespace 

L’option  strategique  maitrise  de  la  violence 
implique  une  surveillance  et  un  controle  tres 
pousses  des  espaces  composant  le  theatre 
d’ operation.  Par  la  maitrise  de  ces  espaces,  le 
commandant  de  niveau  operatif  doit  etre  en 
mesure  de  restreindre  la  liberte  d’action  des 
acteurs  de  violence. 

Dans  les  operations  classiques,  les  armees 
utilisent  l’espace  physique  comme  support  et 
enjeu  essentiels  de  la  manoeuvre.  Dans  les 
operations  de  maitrise  de  la  violence,  ces  rapports 
entre  la  manoeuvre  et  Tespace  perdurent.  Mais  ils 
se  complexifient  par  une  implication  plus 
profonde  de  Taspect  humain  et  organisationnel  de 
Pespace.  Celui-ci  n’est  plus  seulement  support  ou 
enjeu,  il  se  fait  acteur.  Evolutif,  fa^onnable,  il 
devient  une  realite  vivante  impliquee  dans 
Taction.  Sa  connaissance,  les  rapports  des  unites 
de  la  Force  avec  chacune  de  ses  parties, 
engendrent  la  manoeuvre  elle-meme.  Sa  maitrise 
conditionne  la  possibility  d'empecher  Tescalade  de 
la  violence.  L'action  directe  sur  les  belligerants 
constitue  souvent  un  epiphenomene  au  sein  d'une 
gestion  globale  de  Taction  dans  Tespace. 

•  Les  espaces  physiques. 

Les  espaces  aeriens  et  maritimes  forment  des 
milieux  a  peu  pres  homogenes.  Veritables  espaces 
de  combat  dans  les  conflits  symetriques,  ils  sont 
surtout  des  lieux  de  transit  ouvrant  acces  au 
theatre  terrestre  dans  les  operations  de  maitrise  de 
la  violence. 
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L’espace  terrestre  est  plus  complexe  en  raison  de 
son  heterogeneite.  II  est  d’abord  physique  et 
ecologique.  Sa  complexity  est  toujours  apparue 
comme  une  contrainte  majeure  pour  les  operations 
militaries.  II  est  dans  le  meme  temps  malleable, 
contrairement  aux  espaces  aeriens  et  maritimes. 
De  ce  fait,  il  est  a  la  fois  support  et  enjeu  de  la 
manoeuvre.  C’est  sa  morphologie  qui  guide  la 
reflexion  du  stratege  ou  du  tacticien. 

•  Les  autres  dimensions  de  l9espace 

geographique 

A  cote  de  T  analyse  de  1’ espace  physique 
terrestre,  dans  lequel  les  acteurs  de  violence  vont 
s’ organiser  pour  survivre  et  se  developper,  il 
devient  necessaire  de  raisonner  la  manoeuvre 
dans  respace  humain,  lieu  d’identites  et  de 
cultures,  sources  des  antagonismes  territoriaux,  et 
dans  L  espace  structured  lieu  d’organisation  de  la 
vie  politique  et  economique. 

L’espace  humain  s’incame,  en  effet,  dans  la 
notion  de  territoire,  espace  de  vie,  de  rationalite  et 
d’intuitions.  Cette  notion  regroupe  :  l’espace 
vecu,  lieu  social  d’une  communaute,  et  1’espace 
de  representation,  ensemble  des  aspirations 
historiques  et  geographiques  de  cette 
communaute. 

Insistons  sur  ce  dernier  point.  Il  est  frequent 
qu’une  communaute  soit  insatisfaite  de  l’espace 
physique  ou  politique  occupe.  Comprendre  la 
vision  qu’elle  entretient  de  ce  que  devrait  etre  son 
territoire  ideal  permet  de  penetrer  la  dimension 
psychologique  du  conflit  et  de  comprendre  les 
symboles  et  les  valeurs  attaches  au  territoire  pour 
lequel  seront  consentis  les  sacrifices  communs,  et 
au  nom  duquel  sera  justifiee  la  violence  la  plus 
barbare.  Elle  permet  aussi  de  prevoir  les  lieux 
d’affrontement  a  venir  qui  correspondent 
generalement  a  des  positions  geographiques  cles 
ou  des  lieux  symboliques. 

L’espace  structurel  comprend  Tespace  politico- 
administratif,  lieu  des  luttes  de  legitimite,  et 
T  espace  economique,  domaine  des  flux 
d’ appro  visionnement,  d’energie,  d’armements,  de 
financement  des  activites  de  guerre. 

•  Les  reseaux. 

Les  reseaux  constituent  le  tissu  reel  d'un  territoire. 
Ils  rendent  vivantes  les  relations  entre  points  et 
lignes  de  chaque  espace  par  l'intervention 
d'acteurs  humains  les  utilisant  ou  les  creant.  Ces 
relations  sont  variables  et  organisent  les  flux 
circulant  entre  les  points.  Il  sera  toujours 
necessaire,  pour  une  force  militaire  qui  agit  dans 
l'espace  terrestre,  dhdentifier  les  reseaux  et  d'en 
definir  les  relations. 


Les  reseaux  de  Tespace  physique  sont 
facilement  identifiables.  Ils  tiennent  une  place 
importante  dans  Tanalyse  des  facteurs  de  Taction 
militaire  et  dans  T elaboration  de  la  manoeuvre,  en 
termes  de  mobilite,  contre-mobilite  et  points  cles 
a  controler.  Ils  comprennent  principalement  les 
reseaux  de  communication  naturels  dictes  par  le 
relief. 

Les  reseaux  de  Tespace  structurel  sont  aussi 
facilement  identifiables  lorsqu'ils  sont  physiques. 
Ce  sont  des  reseaux  de  circulation  de  biens 
materiels  ou  de  personnes.  Ils  sont  controlables  au 
niveau  des  points  de  communications  (gares, 
aeroports,  carrefours...)  ou  au  niveau  des  points  de 
production  et  de  controle  (energie,  eau).  Leur 
securite  globale  est  plus  difficile  a  garantir.  Les 
reseaux  d'information  constituent  une  categorie  a 
part.  Ils  sont  localisables  en  termes  de  moyens 
materiels  (television,  radio,  bases  de  donnees 
informatiques,  etc.),  mais  beaucoup  plus  difficile 
a  cemer  quand  il  s’agit  de  reseaux  d’influence.  Il 
en  est  de  meme  des  reseaux  financiers 
institutionnels  ou  informels  (diasporas). 

Les  reseaux  humains  sont  moins  facilement 
identifiables,  car  tres  mouvants.  Un  groupe  riest 
jamais  une  collection  de  pions  independants  les 
uns  des  autres.  Il  constitue  un  reseau  qui  s’appuie 
sur  des  infrastructures  materielles  et  qui  dispose 
de  composantes  sociales.  Certains  des  rapports 
qu’il  tisse  sont  provisoires  et  disparaissent 
rapidement,  tandis  que  d'autres  sont 
institutionnalises  :  organisation,  partis,  eglises, 
associations,  etc.  Les  problemes  de  conscience 
collective  sont  done  eminemment  geographiques. 
Raisonner  en  terme  de  reseau  signifie  raisonner  en 
terme  d'interactions.  Dans  le  cadre  d’operations  de 
maitrise  de  la  violence,  la  connaissance  de  ces 
reseaux  humains  est  indispensable.  Le  controle  de 
certains  d’entre  eux  se  revele  facteur  de  succes 
important.  La  dissolution  de  quelques-uns  est 
parfois  necessaire,  tels  les  reseaux  de  financement 
de  factions,  les  reseaux  logistiques,  les  reseaux  de 
circulation  des  armements. 

Ancres  au  sein  des  populations,  lies  a  Texistence 
du  territoire  en  tant  qu'espace  vecu,  ces  reseaux 
constituent  une  des  cles  de  Texecution  de  la 
mission. 

3.  La  maitrise  de  T information 

La  capacite  de  communiquer  et  d'echanger  des 
informations  est  au  coeur  meme  de  la  resolution 
des  situations  conflictuelles.  Elle  requiert  la 
maitrise  complete  des  reseaux  de  communication, 
des  echanges  effectues  et  des  effets  de  ces 
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echanges  sur  le  contexte.  La  situation  conflictuelle 
peut  etre  consideree  comme  une  perturbation  d’un 
vaste  reseau  de  reseaux  dont  il  faut  retablir  la 
stabilite  ou  en  creer  une  nouvelle.  La  maitrise  de 
rinformation  au  sein  de  ces  reseaux  est  done  un 
moyen  d'action  primordial  pour  ceux  qui  sont 
charges  de  ramener  la  paix  soit  par  la  maitrise  de 
la  violence  soit  par  des  actions  de  force. 

•  L 9 information  est  action  et  donne  du  sens 
L’information  est  devenue  action.  Elle  cree  le 
contexte  lui-meme,  le  fait  evoluer  et  ne  se 
distingue  de  faction  en  general  que  parce  qu'elle 
vise  plus  directement  le  plan  des  representations. 
Le  contexte  est  la  cible  meme  des  actes  de 
communication.  Ceux-ci  le  precisent,  Tevaluent, 
le  transforment. 

•  But  de  la  maitrise  de  rinformation 
Maitriser  rinformation,  c’est  ainsi  acceder  a  une 
certaine  transparence  du  contexte  en  identifiant 
1 ’ensemble  des  reseaux  dans  les  differents 
espaces  ;  en  deduire  une  connaissance  la  plus 
precise  possible  de  la  situation ;  anticiper  les 
situations  a  venir  et  1’evolution  du  contexte  dans 
ses  differents  reseaux  et  espaces  ;  decider  d’une 
strategic  ou  d’une  manoeuvre  grace  aux  moyens 
d’aide  a  la  decision  ;  conduire  les  actions  dans  les 
champs  physiques  grace  a  rinformation 
numerisee  et  donner  du  sens  au  contexte  en 
utilisant  la  communication  operationnelle,  la 
communication  mediatique,  les  operations 
psychologies  si  necessaire.  L’ensemble  de  ces 
activites  converge  vers  la  situation  future 
recherchee,  definie  au  moment  de  la  decision 
d’engagement  de  la  Force.  C’est  celle-ci  qui 
donne  le  sens  de  l’action  et  de  la  communication. 
Mais  en  corollaire,  maitriser  l’information,  c’est 
aussi,  selon  le  type  d’operations,  opacifier  le 
contexte  pour  empecher  adversaire  ou  belligerant 
d’en  acquerir  une  claire  connaissance,  de  le 
comprendre  et  d’ anticiper  les  actions  menees  ; 
assurer  la  sauvegarde  des  systemes  et  la  securite 
des  informations  qui  y  circulent  (systemes  de 
recueil  de  1’ information,  de  communication, 
d’aide  au  commandement,  systemes  d’arme) ; 
enfin,  empecher,  dans  les  champs  immateriels,  le 
developpement  de  “  contre-sens  ”  favorables  a 
ceux-ci. 

•  Le  renseignement 

La  notion  de  "transparence  du  champ  de  bataille" 
qui  hante  les  etats-majors  depuis  quelques  annees 
en  raison  des  capacites  d'acquisition  des  systemes 
militaires,  ne  doit  pas  faire  perdre  de  vue  que, 
dans  le  contexte  de  la  maitrise  de  la  violence,  le 
probleme  de  Tacquisition  du  renseignement  est 
assez  different.  Quels  renseignements  doivent  etre 


recherches  ?  II  s'agit  bien  de  toujours  chercher 
des  indices  et  des  donnees,  mais  sur  quoi  ?  II  peut 
ne  pas  y  avoir  ^organisations  au  sens  de 
structures  armees  ;  s'il  y  en  a,  elles  ne  disposent 
pas  forcement  de  materiels  et  de  personnels 
identifiables.  Leur  fonctionnement  peut  ne  pas 
etre  hierarchise  ou  tout  au  moins  peut  ne  pas  se 
decomposer  en  strates  successives  dont  chaque 
niveau  depend  hierarchiquement  du  niveau 
superieur.  Dans  de  nombreux  cas,  les  acteurs  de 
violence  vivront  et  agiront  au  sein  meme  des 
populations  et  ne  s’en  distingueront  pas  forcement. 
Enfin,  leurs  activites  seront  parfois  difficilement 
identifiables,  car  elles  ne  consisteront  pas  en 
actions  de  combat. 

Le  renseignement  a  change  d'objet,  il  s'est  elargi, 
mais  il  existe  et  demeure  indispensable  a  Taction. 
En  ce  qui  conceme  le  renseignement  operationnel 
et  en  dehors  du  renseignement  concemant  les 
acteurs  de  violence,  Taction  de  la  Force  necessite 
Tacquisition  de  nombreux  autres  types 
^informations.  Ces  informations  sont  celles  qui 
ont  toujours  etc  necessaires  aux  forces  pour  se 
deployer  sur  le  terrain  et  y  agir.  Elles  concement 
principalement  Tespace  physique  et  la  situation 
des  unites. 

.  Le  renseignement  terrain,  necessaire  aux 
deplacements  et  aux  deployments,  comme  aux 
diverses  taches  de  combat  ou  de  contention.  Ce 
renseignement  necessite  de  nos  jours  une  unite 
specialisee  de  geographie  dont  la  tache  sera 
d'etablir  ou  d'ameliorer  les  cartes  numerisees 
indispensables  aux  traitements  des  informations. 
Il  est  egalement  donne  par  d’autres  unites 
specialisees  (detachement  de  reconnaissance  du 
genie)  ou  toutes  armes. 

.  Le  renseignement  meteo. 

.  Le  renseignement  d'infrastructure,  qui  conceme 
les  possibilites  de  deployment  et  de  mouvements, 
ainsi  que  les  conditions  d'execution  de  la 
logistique. 

.  Le  renseignement  de  situation  de  Tensemble  des 
unites  participant  a  la  force. 

Ces  besoins  etant  connus  et  maitrises,  ils  ne  seront 
pas  explicits. 
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Le  renseignement  d’environnement  peut  etre 
considere  comme  distinct  du  renseignement 
operationnel  par  le  fait  qu’il  est  recherche,  acquis, 
dans  des  domaines  autres  que  militaires  et  foumi 
par  des  sources  qui  ne  sont  pas  necessairement 
militaires.  II  conceme  principalement  respace 
humain  avec  la  connaissance  des  communautes 
(en  particulier  les  aspects  psychologiques  qui  y 
sont  lies)  et  structured  c'est-a-dire  politico- 
admmistratif  et  economique. 

Le  renseignement  d’environnement  comprend 
trois  domaines  principaux  : 

•  Le  renseignement  concemant  « la  vie  de  la 
cite  »  :  II  s’agit  la  du  renseignement  necessaire  a 
la  restauration  de  la  normalite  publique  et  de  la 
vie  politique.  II  comprend  les  informations 
necessaires  a  la  comprehension  de  la  vie  politique 
locale  et  des  acteurs  politiques,  les  informations 
sur  les  medias,  les  administrations  les  forces  de 
securite  (police,  gendarmerie,  etc.). 

•  Le  renseignement  concemant  la  vie 
economique  :  II  comprend  les  informations  sur  les 
problemes  ecologiques  (securite  et  pollution),  les 
reseaux  et  systemes  de  communications,  les 
reseaux  d’energie,  les  circuits  commerciaux,  les 
foumisseurs  logistiques. 

Le  renseignement  concemant  la  vie  privee  :  II 
rassemble  toutes  les  informations  necessaires  au 
retablissement  d’une  vie  prive  normale  et 
comprend  des  aspects  tres  varies  tels  que  T  attitude 
psychologique  des  communautes  vis-a-vis  de 
Taction  de  la  Force,  Tetat  des  biens  de  premiere 
necessite,  Tetat  de  sante  et  les  mesures  a  prendre, 
les  campagnes  d’information  a  mener  pour  lutter 
contre  accidents  lies  a  la  situation  de  guerre 
(mines,  snipers,  etc.),  Torganisation  de  Taide 
humamtaire,  etc. 

Conclusion 

En  conclusion,  la  visualisation  de  ce  qu’il  est 
convenu  d’appeler  le  champ  de  bataille,  ne  peut 
plus  etre  con9u  comme  elle  Tetait  dans  les  conflits 
symetriques  oil  des  forces  armees  s’affrontaient. 
L’informatique  est  une  aide  precieuse  et  peut 
permettre  de  prendre  en  compte  instantanement 
les  donnees  multiples  et  variees  dont  on  a  besoin. 
La  difficulty  reside  plus  dans  un  changement 
d’optique  et  de  mentalite  sur  Tobjet  de  la 
focalisation  que  sur  la  mise  en  oeuvre  technique.  II 
appartient  aux  operationnels  de  se  remettre  en 
cause  dans  leur  visualisation  des  informations. 
Les  techniques  maintenant  utilisees  sur  Internet 
devraient  permettre  assez  facilement  d’operer 
cette  evolution  :  recherche  par  mots-cles,  liens 


hypertexte,  utilisation  de  couches  d’information, 
etc. 
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SUMMARY 

In  future  battlespace  scenarios  huge  amounts  of  highly 
dynamic  information  will  be  available  due  to  the 
technical  development  of  sensor,  communication  and 
information  systems.  This  flood  of  available  information 
may  lead  to  mental  overload  of  the  military  commander 
and  cause  a  wrong  mental  model  of  the  battlespace 
situation.  Therefore  advanced  techniques  for  supporting 
the  military  commander  and  displaying  complex  tactical 
situation  data  in  a  clearly  understandable  way  have  to  be 
developed  and  evaluated. 

At  the  Research  Institute  for  Communication,  Infor¬ 
mation  Processing  and  Ergonomics  (FKIE)  of  FGAN  a 
concept  for  preprocessing  and  visualizing  incoming 
tactical  data  and  three-dimensional  geographical  data  has 
been  developed.  The  concept  includes  the  use  of  Virtual 
Environment-technology  as  a  display  system.  This 
“Electronic  Sandtable  (ELSA)”  testbed,  as  described  in 
this  paper,  is  based  on  a  semi-immersive  display 
technology.  It  facilitates  a  plastic  stereoscopic 
visualization  of  three-dimensional  data.  It  has  been 
developed  to  be  used  to  simulate  a  sandtable  as 
commonly  used  by  the  Armed  Forces  for  tactical 
education  and  training. 

This  paper  presents  the  baseline  concept  of  using  VE- 
technology  as  an  advanced  tactical  situation  display.  It  is 
pointed  out  that,  although  the  technology  is 
commercially  available,  research  in  the  area  of 
ergonomics  and  human  factors  is  essential  for  the 
advantageous  use  of  such  a  system.  The  main  ergonomic 
topics  described  in  this  paper  include  the  stereoscopic 
visualization  of  the  geographic  and  tactical  data,  the 
degree  of  abstraction  and  human  operator  interaction 
with  the  virtual  scene  on  the  “Electronic  Sandtable”. 

1  Introduction 

Within  the  scope  of  future  scenarios  there  will  be  a  high 
demand  on  detailed  and  highly  actual  information  in 
military  command  and  control  (C2).  The  demand  will  be 
met  by  complex  information  databanks,  new  sensor 
technology  and  fast  electronic  communication  structures. 
Broad  data  acquisition,  transfer  and  presentation  will 


enable  the  military  commander  to  get  a  variety  of  diverse 
information  about  the  battlefield  situation.  The 
accomplished  information  dominance  is  more  and  more 
considered  to  be  essential  for  a  battlespace  dominance. 

However,  the  massive  quantity  of  information  is  also 
hazardous.  Especially  in  time-critical  situations  when 
tactical  decision  making  under  stress  is  required, 
relevant  information  may  be  overseen  and  a  wrong 
mental  model  of  the  tactical  situation  is  gained.  That 
overload  is  likely  to  be  reduced  by  using  new 
technologies  for  data  preprocessing  and  data 
presentation.  Because  data  presentation  is  of  critical 
importance  in  the  whole  process  of  decision  making, 
ergonomic  research  is  required  to  analyze  the  whole 
process  of  data  presentation,  considering  new  displays 
and  interaction  devices. 

Especially  using  Virtual  Environment  (VE)-technology 
is  promising.  It  was  found  to  have  high  potential  in 
presenting  and  interacting  with  complex  amounts  of 
data.  Therefore  VE  will  increase  the  clearness  and 
intelligibility  of  a  complex  tactical  situation.  The 
situation  scenario  is  not  perceived  as  a  complex  of 
abstract  information  but  as  a  pseudo-realistic  model 
landscape.  This  is  intensified  by  an  intuitive,  easy  to 
learn  interaction  with  the  included  objects. 

2  Definition  of  Virtual  Environment  (VE) 

The  basic  idea  of  generating  and  using  a  computer- 
generated  artificial  reality  was  mentioned  first  in  science 
fiction  literature  at  the  middle  of  the  20th  century.  Due  to 
rapid  development  of  computer  technology  in  the  second 
half  of  the  last  century,  a  partly  realization  of  this  idea 
became  possible.  Nowadays  these  VE-Systems  are 
commercially  available  and  starting  to  be  used  for  a 
broad  range  of  applications  (Alexander  et  al,  1999). 

According  to  Bullinger  et  al  (1997),  Virtual  Environ¬ 
ments  (VE)  describe  the  computer-based  generation  of 
an  intuitively  perceivable  and  experiencable  scene  of  a 
natural  or  an  abstract  environment.  It  is  characterized  by 
capacities  for  multi-modal,  three-dimensional  modeling 
and  simulation  of  objects  and  situations.  A  further 
characteristic  is  the  close  interaction  of  the  human 
operator  with  the  system  . 


Paper  presented  at  the  RTO  HFM  Symposium  on  “Usability  of  Information  in  Battle  Management 
Operations”,  held  in  Oslo ,  Norway ,  10-13  April  2000 ,  and  published  in  RTO  MP-57. 
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In  this  connection.  Virtual  Reality  (VR),  has  been 
defined  by  NATO  HFM-021  (n.n.)  as 

the  experience  of  being  in  a  synthetic 
environment  and  the  perceiving  and  interacting 
through  sensors  and  effectors ,  actively  and 
passively ,  with  it  and  the  objects  in  it,  as  if  they  were 
real  Virtual  Reality  technology  allows  the  user  to 
perceive  and  experience  sensory  contact  and 
interact  dynamically  with  such  contact  in  any  or  all 
modalities.  ” 

This  definition  of  VR  which  is  often  used  as  a  synonym 
to  VE  overlaps  with  VE.  But  whereas  VE  is  application 
oriented,  VR  describes,  strictly  speaking,  a  total  model 
of  the  reality,  including  all  manifold  facets  of  it.  As  this 
is  not  possible  today  and  may  not  be  possible  in  future, 
the  further  article  will  be  use  the  term  VE. 

VE  can  be  divided  into  at  least  three  groups  (Bullinger  et 
al,  1997): 

•  full-immersive  VE  is  characterized  by  a  complete 
replacement  of  the  reality  by  the  virtual  reality.  The 
operator  is  completely  included  in  the  virtual  world 
and  does  not  perceive  any  (visual,  acoustic)  stimuli 
from  the  real  word.  A  head-mounted  display  (HMD) 
is  a  typical  full-immersive  display. 

•  semi-immersive  VE:  The  virtual  scene  is  presented  as 
a  three-dimensional  part  of  the  reality.  The  operator 
perceives  stimuli  from  the  real  world  and  additional 
stimuli  from  the  virtual  world.  He  cannot  distinguish 
between  real  and  virtual  objects.  Typical  semi- 
immersive  VE-Systems  are  workbenches  which  will 
be  described  later,  and  flight  simulators. 

•  Desktop  VE:  The  three-dimensional  scene  is 
presented  on  a  two-dimensional  display  medium.  Just 
interaction  and  navigation  happen  more  intuitively. 
VRML-Browsers  and  Videogames  work  like  this. 

VE-systems  are  on  their  way  of  becoming  used  for 
different  applications.  Further  information  about  military 
applications  is  given  in  Alexander  et  al  (1999). 

First,  VE  is  a  research  topic  itself.  This  involves  basic 
research  studies  in  computer  science  as  well  as  interface 
design  and  ergonomics. 

Secondly  VE  is  used  as  a  research  and  development 
tool  In  this  area  VE  Systems  enable  a  very  intensive  and 
direct  interaction  with  complex  and  abstract  data.  They 
enable  new  kinds  of  rapid  prototyping.  In  an  early  stage 
of  the  design  process  CAD-models  of  products  can  be 
visualized  and  examined  as  if  they  were  real. 
Improvements  can  be  performed  easily  in  real-time  and 
the  effects  can  be  visualized  immediately.  This  bring 
along  advantages  for  the  amount  of  time,  the  quality  and 
the  cost  of  the  development  cycle.  In  Fig.  1  an  example 
for  a  virtual  walk-through  in  the  design  process  of  a 
marine  vessel’s  combat  information  center  (CIC)  is 
shown. 

Another  application  is  teleoperation  and  telepresence. 
This  is  not  limited  to  remote  control  of  unmanned 
systems.  Moreover  the  operator  gets  the  subjective 
feeling  of  really  being  there.  This  may  enable  higher 
situational  awareness  which  is  considered  to  be 
advantageous  for  this  application.  Furthermore, 
teleoperation  is  supposed  to  be  useful  for  control  of 
robots  in  contaminated  areas,  space  or  deep  sea  and 
special  military  purposes  (reconnaissance,  surveillance). 


Finally,  the  area  of  education  and  training  is  a  field  of 
application.  In  contrast  to  conventional  virtual 
simulation  and  simulators  VE-systems  are  more  flexible 
and  adaptive.  This  is  because  one  single  VE-system  can 
be  used  to  simulate  different  types  of  vehicles  or 
aircrafts.  Moreover,  training  of  individual  soldiers’  skills 
as  well  as  team  training  become  possible. 


Fig.  1:  Virtual  combat  information  center  (CIC) 


The  benefits  of  VE  which  have  been  shown  in  current 
approaches  in  these  areas,  make  it  a  promising  tool  for 
further  applications.  One  of  these  applications  is  a 
tactical  situation  display  (TSD)  of  a  Command  & 
Control  system. 


3  Tactical  Situation  Displays  (TSD)  today 

The  basic  function  of  TSDs  is  to  display  the  current 
situation  of  own  and  reconnoitered  enemy  troops  and 
facilities  in  the  operation  area  to  the  commander  of  a 
military  unit.  Moreover  the  TSD  is  used  for  tactical 
planning  of  intended  future  operations.  Quantity  and 
quality  of  situation  data  are  essential  for  an  adequate 
operation  planning  (Grandt  et  al,  1997). 

Today  there  are  basically  two  different  types  of  TSD’s 
used  by  the  strike  forces. 

The  first  one,  shown  in  Fig.  2,  is  a  command  post  in  the 
field.  The  TSD  used  here  works  by  means  of  paper  & 
pencil.  Actual  information  is  transmitted  by  radio  or 
field  telephone  and  drawn  into  a  map.  It  is  obvious  that 
in  time-critical  processes  with  large  amounts  of  rapid 
changing  information  this  leads  to  an  overload  of  the 
operators.  Furthermore,  the  display  may  not  show  actual 
or  valid  information  and  causes  errors  in  decision 
making.  However,  it  brings  along  the  advantage  that  the 
commander  is  in  the  field:  He  gains  high  situational 
awareness,  experiences  the  terrain,  cover,  weather,  etc. 
and  knows  “what  is  really  going  on”  at  that  place. 


Fig.  2:  TSD  at  command  posts  “in  the  field” 
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On  the  other  hand  there  are  TSDs  at  operation  centers , 
as  can  be  seen  in  Fig.  3.  Tactical  situation  data  is 
preprocessed  and  computers  are  used  to  visualize  the 
results. 

The  advantages  of  these  advanced  TSD’s  are: 

•  actuality  of  data,  provided  that  the  communication 
infrastructure  is  fast  enough, 

•  different  views  of  levels  of  data  aggregation  and 

•  possibilities  to  include  additional  battlespace 
information. 

But  this  flood  of  information  may  lead  to  an  information 
overload;  moreover  data  representation  is  still  limited  to 
two  dimensions  and  techniques  of  interaction  with  data 
have  to  be  learnt. 


Fig.  3:  TSD  at  operation  centers 


The  approach  of  using  VE  as  TSD  first  expands  the  two- 
dimensional  visualization  to  three  dimensions.  This 
means  that  height  information  can  easily  be  perceived. 
Additional  elevation  aids,  like  elevation  profiles  or  color 
texturing,  can  be  skipped  and  replaced  by  others  (e.g. 
reconnaissance  photos,  weather  data,  etc. 

The  more  important  thing  is  that  general  interaction  with 
data  is  simplified  and  happens  more  intuitively.  This 
facilitates  an  experience  of  the  tactical  situation  and  the 
generation  of  a  correct  mental  model.  In  an  ideal  VE- 
system  the  computer  is  not  realized  as  an  active  entity, 
but  becomes  an  invisible  assistant  which  knows  about 
user  intentions  and  supports  him  (Alexander  et  al., 
1997).  Therefore  operator  workload  is  supposed  to  be 
reduced  and  situational  awareness  to  be  increased. 

4  Approaches  of  VE-Technology  in  C2 

The  amount  of  studies  and  applications  in  the  area  of  VE 
and  VE-technology  has  increased  rapidly  recently.  But 
whereas  VE  is  close  to  become  applicable  in  research 
and  development  and  for  single  training  applications, 
studies  considering  the  specific  use  of  VE  in  C2  have  just 
begun.  Therefore  knowledge  in  this  area  is  limited  and  a 
lot  of  projects  are  in  a  conceptual  phase. 

Most  research  studies  and  projects  in  this  area  have  been 
started  in  the  past  two  years.  Because  of  ongoing 
development  in  this  area  this  is  only  a  brief  overview. 
Detailed  information  is  given  in  Alexander  et  al.  (1999). 

Generally  speaking,  the  approaches  can  be  divided  into 
two  groups.  The  first  group  consists  of  concepts  and 
long-term  programs  including  VE-components.  This  is  a 
top-down  approach  which  takes  place  at  high  political 


level  and  typically  application-oriented.  The  second 
group  is  characterized  by  specific  VE-projects  and 
laboratories.  Consequently  it  follows  a  bottom-up 
approach  and  is  presentation-  and  technology-oriented. 
Fortunately,  there  are  links  between  both  so  that  they 
meet  and  synergetic  effects  exist. 

The  Swedish  ROLF  ( Mobile  Joint  Command  and 
Control  System  2010)  is  a  long-term  program.  Its  goal  is 
to  determine  new  possibilities  for  military  commanders 
of  using  VE-Technology  in  mobile  command  posts. 
ROLF  describes  requirements  for  situational  awareness, 
decision  making  and  support,  work  methodology  and 
organization  of  military  crew  and  staff.  The  main  idea  is 
to  use  modem  methods  and  technology  to  help  a  group 
of  operators  in  difficult  situations  with  complex,  time- 
critical  decision  making.  ROLF  includes  the  Aquarium 
as  TSD  which  is  a  semi-immersive  VE-system.  The  TSD 
is  used  to  visualize  positions  of  own  and  enemy  troops, 
positions  of  important  institutions,  terrain  and  weather 
data  in  different  views.  Data  preprocessing  is  used  to 
select  the  data  displayed  and  ensures  that  only  important 
information  is  visible  (Sundin,  1996). 

Especially  the  realization  that  in  future  battle  scenarios 
all  actions  of  the  military  commander  will  be  in  an 
unclear,  vague  environment  and  the  importance  of  an 
information  dominance  led  to  the  development  of  the 
Command  Post  of  the  Future  Program  (CPoF)  of 
DARPA  (1998).  The  program’s  goal  is  to  accelerate  the 
decision  making  process  with  ongoing  reduction  of  the 
staff.  Therefore  new  technology  is  needed  to  make 
maximum  use  of  the  whole  human  perceptory  system  in 
order  to  transmit  maximum  amount  of  information.  This 
includes  an  interactive,  three-dimensional  visualization, 
three-dimensional  interaction  with  computer-generated 
objects,  presentation  of  inaccuracy  and  probability, 
integration  of  dynamic  factors,  three-dimensional 
symbolic,  integration  of  natural  language  processing  and 
integration  of  knowledge-based  systems. 

The  second,  more  technology-oriented  group  of 
approaches  is  larger.  Institutions  and  laboratories 
working  in  this  area  use  different  VE-technology.  The 
technology  is  often  reconfigured  to  be  used  for  different 
research  projects  and  experiments. 

The  US  Battle  Command  Battle  Lab  (BCBL)  performs 
conceptional  studies  as  well  as  experimental  analyses  in 
a  VR-laboratory.  One  goal  is  to  develop  a  technology  for 
a  multi-media,  scene-based  application  in  education  and 
training  for  organization  and  staff  functions.  This  system 
shall  be  connected  to  the  internet  to  increase  the  range  of 
application  (Heredia,  1999). 

At  the  US  Naval  Research  Laboratory  (NRL)  an 
advanced  battle  planing  and  management  system  has 
been  developed.  The  system  works  with  a  semi- 
immersive  display  and  enables  multi-modal  interaction. 
It  was  found  to  be  very  suitable  for  virtual-prototyping, 
interactive  mission  planing  and  increasing  situational 
awareness  (NRL,  1997). 

Similar  approaches,  like  Mirage  of  the  Army  Research 
Lab  (ARL)  (1ST,  1997),  the  Visualization  Architecture 
Technology  (VAT)  of  the  Crewstation  Technology 
Laboratory  (CTS)  (Achille,  1998)  or  the  Electronic  Sand 
Table  of  MITRE  Corp.  (MITRE,  1998)  also  use  a  semi- 
immersive  VE-technology,  as  described  further  on. 
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Other  approaches  use  full-immersive  VE  or  desktop- VE 
respectively  (Dockery  &  Hill,  1996;  Morgenthaler  et  al, 
1998). 

5  The  Electronic  Sandtable  (ELSA) 

The  Electronic  Sandtable  has  been  developed  as  an 
advanced  display  for  tactical  information  in  mission 
planing,  control  and  rehearsal.  The  concept  is  based  on 
the  sandtable  metaphor.  The  military  sandtable,  as 
shown  in  Fig.  4,  consists  of  a  sandy  model  landscape 
with  simplified  objects  representing  woods,  buildings, 
points  of  interest  or  military  units.  It  is  broadly  used  in 
military  education  and  training. 


Fig.  4:  Sandtable  in  military  education 


But  the  traditional  sandtable  is  static;  all  changes  of 
deployment  have  to  be  done  manually.  Each  change  of 
region  is  very  time-consuming  and  has  also  to  be  done 
manually.  Moreover  the  accuracy  for  representing  real 
geographic  data  is  poor. 

It  is  intended  to  model  the  sandtable  by  means  of  a  VE- 
system.  This  way  dynamics,  real-time  interaction  and 
changes  of  the  point-of-view  can  be  included  while  the 


benefits  of  the  real  sandtable  remain. 

For  this  purpose  geographic  data  and  tactic  data  have  to 
be  visualized  stereoscopically.  It  is  intended  to  create  a 
model  landscape,  in  which  dynamic  battle  scenario  is 
included.  Furthermore  additional  functionality  is  added, 
e.g.  visibility,  range  of  weapon  systems,  etc. 

5.1  Structure  and  Technical  Implementation 

The  Electronic  Sandtable  has  been  implemented  as  a 
testbed  at  the  Research  Institute  for  Communication, 
Information  Processing  and  Ergonomics  (FKIE).  A  draft 
of  the  technical  setup  is  shown  in  Fig.  4. 

Because  of  the  large  size  of  geographic  databanks  and 
the  need  for  real-time  interaction,  the  underlying 
structure  has  been  arranged  in  two  stages  (Alexander  et 
al,  1997). 

The  first  stage  is  executed  offline.  In  this  stage  the  scene 
graph  is  determined.  The  scene  graph  is  a  hierarchically 
ordered  databank  of  all  polygons  included  in  the  visible 
scene. 

Originally  all  data  is  separated  in  different  databanks, 
which  means: 

•  digital  terrain  elevation  data  (DTED), 

•  digital  feature  analyses  data  (DFAD), 

•  textures  (e.g.  reconnaissance  pictures), 

•  single  geometric  objects  (buildings,  tanks,  airplanes) 
and  their  attributes. 

In  a  semi-automatic  process  data  and  objects  are 
selected,  integrated  and  re-ordered  with  respect  to 
maximum  rendering  performance.  This  databank  is 
called  the  scene  graph.  Afterwards  the  structure  of  the 
scene  graph  stays  constant  without  any  changes. 

In  the  second  stage  additional  data  is  constantly  added 
and  the  scene  graph  is  visualized  online.  The  additional 
data,  i.e.  tactical  situation  data  and  data  from  external 
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Fig.  4:  Structure  of  the  Electronic  Sandtable  (ELSA) 
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data  sources,  is  linked  to  objects  of  the  scene  graph. 
Additional  input  of  external  data  using  different 
protocols  (DIS,  HLA)  shall  also  become  possible  in 
future.  The  incoming  data  controls  position  and  status  of 
military  units.  Additional  data,  like  actual  situation 
videos  or  information  of  knowledge  databanks,  can  also 
be  included. 

After  that  the  rendering  subsystem  selects  the  visible 
subset  of  the  scene  graph.  Out  of  this  two  separate 
projections  are  calculated  and  written  into  the  two  frame 
buffers.  Then  both  frame  buffers  are  visualized 
alternately  on  a  horizontal  plane. 

The  human  operator  interacts  with  the  scene  by  means  of 
different  interaction  devices.  The  inputs  serve  as 
commands  which  affect  the  objects  of  the  scene  graph. 
The  actions  are  logged  for  later  analysis. 

The  operator  is  able  to  select  different  visible  areas  for 
navigation.  The  borders  of  the  area  serve  as  one  input 
variable  of  the  rendering  subsystem.  Additionally  each 
of  the  operator’s  movements  is  tracked  by  a  head- 
tracker.  The  position  output  of  the  tracker  is  another 
input  variable  of  the  rendering  subsystem  for  new 
projection  calculation. 

5.2  Generation  of  the  Scene  Graph 

The  scene  graph  is  the  output  of  the  first  stage  described 
in  chapter  5.1.  The  generation  process  itself  is  offline 
and  semi-automatic.  As  far  as  possible  COTS-products 
are  used. 

The  process  is  divided  into  a  data  selection, 
preprocessing  and  optimizing  phase. 

Preprocessing  and  optimizing  are  necessary  because 
terrain  and  feature  data  are  generated  from  geographic 
databanks.  These  databanks  were  designed  with  regard 
to  different  requirements  which  makes  them  unsuitable 
for  a  real-time,  realistic  visualization. 

5.2.1  Data  Selection 

In  the  first  step  an  area  of  interest  is  selected  and  the 
relating  terrain  (DTED)  and  feature  (DFAD)  data  is 
extracted.  Additionally,  links  between  features  and 
geometric  objects  are  defined.  Afterwards  the  selected 
data  is  saved  in  a  temporary  buffer  which  has  to  be 
preprocessed  and  optimized  for  visualization. 

The  geographic  data  available  is  divided  into  (Helmuth, 
1996): 

•  Raster  data:  pixel  data  (e.g.  scanned  paper  maps), 

•  Picture  data:  geo-referenced  or  non-referenced  aerial 
or  satellite  photos. 

•  Vector  data:  surfaces  (e.g.  woods,  lakes),  lines  (e.g. 
streets,  rivers)  or  points  (e.g.  power  poles,  points  of 
interests,  bridges,  towers)  with  the  position  of  their 
bases  and  attributes.  For  visualization  vector  data  is 
linked  to  detail  objects. 

•  Matrix  data:  data  in  matrix  format  of  a  specific 
resolution.  Usually,  terrain  data  is  organized  like  this. 

Geometric  objects  are  components  which  are  linked  to 
geographic  data,  especially  geographic  vector  data,  and 
tactical  situation  data.  They  include  a  geometric 
description  of  the  object  (e.g.  tanks,  airplane)  and 
additional  information  (e.g.  unit  status,  damage  reports, 
etc.).  At  the  stage  of  real-time  visualization  they  are 


shown  at  the  position  given  either  by  the  geographic 
data  or  the  tactical  situation  data.  General  geometric 
objects  are  often  automatically  constructed.  However, 
more  sophisticated  models  have  to  be  designed  manually 
either  by  a  CAD-program,  a  modeling  software  or  out 
of  an  object  databank. 

For  later  selection  operations  attributes  have  to  be 
added.  Attributes  can  be  divided  into  geometric  and 
general  attributes.  While  geometric  attributes  (length, 
width,  height)  are  the  same  for  each  object,  general 
attributes  are  dependent  on  the  kind  of  application.  Such 
attributes  might  be  population  data  (sociology),  pollutant 
emission  (environment)  or  tactical  information 
(military). 

5.2.2  Data  Preprocessing 

Consistency  and  integrity  are  highly  important  criteria 
for  databanks.  If  datasets  of  more  than  one  databank  are 
merged,  contradicting  data  might  emerge  and  cause 
errors.  Those  errors  are  based  on  errors  or  inaccuracies 
in  the  original  databanks,  different  data  resolution  or 
different  actuality  of  data  acquisition. 

As  soon  as  consistency  and  integrity  is  proved,  the 
process  of  merging  terrain  and  feature  data  starts. 
Geometric  objects  are  appended  and,  if  necessary, 
adjusted  to  ground  level. 

Finally  the  triangulation  process  starts  and  determines 
polygons  for  visualization. 

5.2.3  Data  Optimizing 

For  real-time  visualization  the  amount  of  rendered 
polygons  has  to  be  minimal.  Therefore  the  databank 
system  transfers  only  information  about  the  visual 
subset.  Non-visible  parts  outside  the  field  of  view  are 
clipped. 

For  further  reduction  the  databank  is  re-organized  and 
the  scene  graph  is  tiled.  In  the  visualization  process  the 
distance  to  the  point  of  view  sets  the  level  of  complexity 
for  each  tile. 

Different  levels  of  complexity,  also  called  levels  of 
detail  (LOD),  are  another  technique  to  reduce  polygons. 
LOD  means  more  than  one  representation  of  different 
levels  of  complexity  (different  amount  of  polygons)  for 
the  same  subset.  This  means,  if  a  subset  gets  closer  to 
the  point  of  view,  a  higher  LOD  with  more  polygons  is 
visualized. 

Using  these  techniques  data  is  re-organized  with  regard 
to  visualization  issues.  The  output  of  this  process  is  the 
scene  graph  which  is  visualized  in  real-time  in  the 
second  stage. 

5.3  Display  Technology 

The  display  technology  used  for  three-dimensional 
visualization  is  a  semi-immersive  virtual  workbench. 
This  concept  has  originally  been  developed  by  Kruger  & 
Frohlich  (1992).  The  baseline  concept  is  shown  in  Fig. 
5.  Today  it  is  used  for  various  applications. 

A  projector  projects  two  computer-generated,  time- 
alternated  pictures  onto  a  mirror.  The  mirror  reflects 
them  to  a  horizontal  focussing  screen.  By  using  shutter 
glasses,  i.e.  LCD-glasses  shading  each  side  alternately 
synchronous  to  the  projection,  the  operators  perceive 
two  separate  pictures  for  the  right  and  the  left  eye.  The 
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synchronization  works  by  an  emitter  sending  infrared 
signals  synchronously  to  the  picture  projected. 


Projector  Virtual  Workbench 


Fig.  5:  Principle  of  a  semi-immersive  virtual  workbench 

Finally,  both  pictures  perceived  are  fusioned  by  the 
cerebrum  to  a  single,  three-dimensional  model. 

6  User  Interface 

The  design  of  the  user  interface  of  VE-systems  has  been 
found  to  be  one  of  the  main  criteria  of  quality  for  its 
application.  The  Electronic  Sandtable  serves  as  the 
interface  between  the  real  environment  on  the  one  hand 
and  the  virtual  scene  on  the  other  hand.  Moreover  it  uses 
a  different  metaphor  than  the  desktop-metaphor  used  in 
various  computer  applications.  Therefore  new 
interaction  techniques  and  procedures  have  to  be 
developed,  analyzed  and  optimized  according  to  a  high 
performance  of  the  human-computer-system  (Alexander, 
1999). 

This  includes  visualization,  interaction  and  cooperation 
in  a  virtual  scene. 

6.1  Visualization 

A  realistic,  three-dimensional  visualization  of  terrain 
data  has  to  consider  the  physiological  procedures  of 
visual  depth  perceiving.  These  procedures  have  been 
studied  extensively,  and  several  different  hypothesis  for 
depth  perceiving  exist. 

Each  hypothesis  postulates  the  existence  of  depth  cues. 
The  classic  depth  cues  will  be  summarized  later  in  this 
chapter.  Especially  the  depth  cues  of  stereoscopic 
disparity  and  parallax  are  of  critical  importance  for  the 
application  of  the  Electronic  Sandtable. 

A  computer-based  visualization  has  to  take  into  account 
different  depth  cues.  For  stereoscopic  visualization 
different  viewing  models  exist.  The  common  models 
will  be  presented  in  this  chapter  as  well. 

6.1.1  Visual  Perception 

The  physiological  visual  system  consists  of  the  eye  as 
sense  organ  for  stimulus  acquisition,  the  optic  nerve  for 
stimulus  transfer  and  the  optic  center  of  the  cerebrum  for 
stimulus  processing. 

According  to  Schmidt  &  Thews  (1995)  the  human  eye 
can  be  divided  into  two  subsystems: 

•  Subsystem  1  is  responsible  for  the  refraction  of 
incoming  light.  Its  main  components  are:  Iris  (control 
of  incoming  light  intensity),  lens  (refraction), 
vitreous  body  (stability)  and  diverse  muscles 
(adjustment). 


•  Subsystem  2,  jointly  with  the  central  nervous  system, 
transfers  the  light  to  stimulus  signals  of  nerve  cells.  It 
consists  of  the  retina  with  its  two  different  light 
receptors. 

The  stimuli  are  transferred  via  the  optic  nerve  to  the 
optic  centers  of  the  cerebrum.  Here  the  optic  sensing  and 
recognition  takes  place. 

Visual  perception  is  generally  based  on  three  stages  of 
perception  (Kelle,  1994): 

The  first  stage  is  an  egocentric  perception  of  the  own 
person.  This  allows  a  separation  of  objects  of  the  own 
body  and  other  objects,  making  possible  to  determine  the 
own  position  with  regard  to  other  objects  and  an 
absolute  depth  perception . 

The  next  step  is  a  comparision  of  the  objects  in  the 
environment,  allowing  a  relative  depth  perception . 

Finally  memory,  experience  and  internal  processing 
mechanism  lead  to  depth  cues  being  fundamental  for 
spatial  perception. 

6.1.2  Depth  Cues 

Depth  cues  are  visual  system  cues  which  enable 
perceiving  of  spatial  dependencies  (Hodges,  1992; 
Schmidt  &  Thews,  1995).  The  can  be  divided  into 
monocular  and  binocular  cues. 

Monocular  cues  are  valid  for  perception  with  one  eye 
only. 

The  main  monocular  cues  are: 

•  perspective, 

•  difference  in  size, 

•  known  dimensions  of  objects, 

•  shading, 

•  light  and  shadow, 

•  accommodation. 

The  binocular  depth  cues  require  the  total  binocular  eye 
system.  They  influence  the  perception  of  short  to 
medium  distances. 

Traditional  binocular  depth  cues  are: 

•  convergence:, 

•  disparity  and  parallax. 

Additionally  to  these  static  cues  there  are  dynamic  cues 
which  have  large  influences  on  the  depth  perception  for 
medium  distances  (17  -29  m)  (Kelle,  1994). 

6.1.3  Disparity  and  Parallax 

Disparity  and  parallax  have  a  large  influence  on  depth 
perception  and  are  the  main  depth  cues  for  stereoscopic 
visualization.  Therefore  they  are  described  more 
detailed. 

The  distance  between  both  eyes  leads  to  different 
representations  of  an  object  on  the  retina  of  the  right  and 
the  left  eye.  Both  eyes  perceive  the  object  with  a 
different  perspective.  The  difference  between  both 
pictures  is  described  by  the  disparity. 

If  an  object  is  focussed,  it  is  represented  at  the  fovea  of 
both  eyes.  A  round  spatial  surface  exists  (horopter), 
representing  all  objects  on  it  on  corresponding  retina 
areals.  Objects  at  positions  different  from  the  horopter 
are  represented  at  non-corresponding  retina  areas.  If  the 
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distance  from  the  horopter  is  not  too  large,  the  cerebrum 
fusions  the  right  and  left  picture  to  a  three  dimensional 
model.  If  it  is  too  large,  disturbing  double  pictures  are 
perceived  (Schmidt  &  Thews,  1995). 

Disparity  is  a  mathematical  dimension  and  cannot  be 
determined  practically.  Therefore  the  dimension  of  the 
stereoscopic  parallax  has  been  introduced.  For  this  a 
reference  level  has  been  used  which  is  parallel  to  the 
eyes’  level  and  runs  through  the  fixation  point. 

Parallax  has  been  defined  as  (Helmholtz,  1910,  ref.  in: 
Kelle,  1994): 


p  =  ba  xax 


e*  t  +  e 


p  =  parallax 

ba  =  inter  ocular  distance 

a  =  distance  eyes  /  reference  level 

e  =  distance  reference  level  /  object 

t  =  distance  eyes  /  object  ( =a+e  ) 

Parallax  is  also  a  dimension  for  depth  separation  and 
depth  perception.  Therefore  it  is  deduced  that  depth 
perception  decreases  with  square  distance.  Furthermore 
it  increases  linearly  with  inter  ocular  distance. 

According  to  Kelle  (1994),  stereoscopic  disparity  and 
parallax  has  been  found  to  be  useful  only  for  near  and 
medium  distance  (maximum  of  6-9  m). 

Visualization  of  geographic  data  of  large  scale  means  a 
large  distance  between  eye  point  and  surface.  It  can  be 
concluded  that  exact  modeling  means  that  parallax  and 
stereoscopic  depth  perception  will  be  very  low.  Instead 
inter  ocular  distance  has  to  be  modified  to  several  meters 
evoking  a  higher  stereoscopic  depth  perception. 


6.1.4  Stereoscopic  Projection  Models 

For  three-dimensional  stereoscopic  visualization  three 
different  projection  models  are  commonly  used.  Their 
baseline  geometry  is  illustrated  in  Fig.  6. 


(a) 


(b) 


(c) 
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Fig.  6:  3  projection  models:  (a)  parallel  projection, 
(b)  rotated  projection,  (c)  window  projection 


In  Computer  Aided  Design  (CAD),  aerial  photo  analysis 
and  for  head-mounted-displays  (HMD)  projection 
models  with  parallel  line  of  sights  are  used,  as  shown  in 
Fig.  6  (a).  They  are  based  on  the  assumption  of  a  center 
eye-point  perpendicular  to  the  projection  plane.  Right 
and  left  projection  are  calculated  by  using  offset  values 
and  parallel  shifting  the  projection  right  and  left.  The 
disadvantage  of  this  model  is  that  the  scene  can  only  be 
visualized  underneath  the  projection  plane.  This  is 
inconvenient  for  the  concept  of  the  Electronic  Sandtable, 
because  the  scene  would  always  be  located  beyond  hand 
range.  Another  disadvantage  is  clipping  at  the  borders  of 
the  display  as  there  are  missing  visual  information  for 


either  the  right  or  the  left  eye.  Especially  at  large 
displays  this  is  very  irritating  for  operators. 

Fig.  6  (b)  shows  the  geometry  of  a  projection  model 
using  rotated  line -of- sights.  Here  the  projections  are 
rotated  in  the  way  that  both  lines-of-sight  meet  in  the 
projection  plane.  The  lines-of-sight  are  not 
perpendicular  to  the  projection  plane.  The  concept 
remains  the  same,  no  matter  if  the  scene,  the  eye  points 
or  both  are  rotated.  It  enables  a  visualization  underneath 
and  as  well  as  above  the  projection  plane.  There  are  no 
irritating  effects  on  the  borders  of  the  display  either.  But 
because  of  the  special  geometry,  an  error  of  vertical 
parallax  appears.  This  can  be  observed  especially  at  the 
borders  of  the  display,  where  both  lines  meet  at  a  point 
above  the  projection  plane.  This  leads  to  a  “winding”- 
effect  and  the  scene  seems  to  be  projected  on  a  cylinder 
rather  than  a  plane.  The  error  is  perceived  especially  on 
large  displays. 

The  last  projection  model  uses  window  projection , 
which  means  that  two  windows  are  introduced  through 
which  the  virtual  scene  is  perceived.  The  windows  are 
positioned  in  the  same  level  as  the  projection  plane. 
Both  lines-of-sight  meet  at  the  projection  plane  and 
remain  perpendicular  to  it.  This  is  correct  for  the  middle 
and  the  border  of  the  projection  area.  In  this  model, 
stereoscopic  parallax  is  only  dependent  on  the  distance 
to  the  display  and  no  vertical  parallax  is  introduced. 

This  model  is  used  for  the  Electronic  Sandtable.  As 
shown  in  Fig.  7,  the  model  for  each  eye  is  described  by 
an  asymmetric  pyramid.  This  means,  the  perpendicular 
line  through  the  top  does  not  meet  the  center  of  the 
pyramid  basis. 


Fig.  7:  Right  and  left  asymmetric  projection  pyramid 
and  boundary  surfaces  (clipping  planes) 


For  each  projection  six  parameters  are  used  to  identify 
the  pyramid.  They  include  the  values  for  front,  back,  top, 
bottom,  left  and  right  clipping  plane.  These  values  are 
calculated  by  x,y,z-position  of  both  eye-points,  scale 
factor  and  the  display  size  as  input. 

Pilot  experiments  have  shown  good  results  for  this 
projection  model.  Only  little  perspective  error  due  to 
tracking  of  the  real  eye  position  was  determined.  In 
future,  this  error  will  be  minimized  by  calibrating  the 
tracking  equipment. 

6.1.5  Future  Research  in  Visualization 

So  far  only  real-size  shapes  have  been  visualized.  In 
future  geographic  data  of  different  scales  will  be  used. 
To  evoke  a  stereoscopic  depth  perception,  an  adaption  of 
the  scale  factor  for  elevation  as  well  as  the  dimension  of 
inter  ocular  distance  is  necessary. 
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However,  the  adaption  may  lead  to  either  too  flat  or  too 
heigth  depth  perception.  Both  is  not  wanted  and 
therefore  research  studies  will  be  done  concerning  the 
correct  determination  of  both  parameter. 

Another  research  topic  is  the  maximum  vertical  range  of 
the  display.  The  display  technique  causes  contradicting 
depth  information,  because  both  eyes  accomodate  on  the 
projection  plane,  but  fixate  an  object  closer  or  more  far 
away.  However,  if  the  virtual  scene  is  too  close,  parallax 
becomes  too  large  and  the  cerebrum  cannot  fusion  both 
pictures.  Therefore  another  research  topic  will  be  to 
determine  the  maximum  useful  vertical  display  range 
and  the  variability  of  human  sense  perceiving. 

Pilot  experiments  in  this  area  have  been  started  and  are 
currently  going  on. 

6.2  Interaction 

Interaction  with  the  databank  means  navigation  in  the 
scene  and  manipulation  of  virtual  objects.  For  both 
subgroups  procedures  (software)  and  interaction  devices 
(hardware)  have  to  be  designed,  evaluated  and  analyzed 
according  to  the  application. 

6.2.1  Navigation 

Navigation  can  be  divided  into:  Navigation  within  the 
databank  system  and  navigation  within  space. 

The  first,  navigation  within  the  databank  system , 
encloses  different  procedures  for  search  and  selection  of 
datasets.  This  is  a  general  topic  and  is  not  of  special 
interest  for  VE  systems.  Therefore  it  will  not  be 
described  in  this  paper. 

Navigation  within  space  means  a  change  of  position  or 
orientation  of  the  observer.  This  is,  generally  speaking,  a 
modification  of  display  area.  A  first  implementation  sets 
a  starting  position  and  orientation  for  the  observer.  Both 
can  be  modified  by  user  input.  There  are  many 
possibilities  for  designing  the  navigation  procedure  of 
the  graphic  user  interface  (GUI). 

One  is  an  adaption  of  the  procedures  used  by  MS 
Windows®  or  OS  Motif®.  An  example  of  this  can  be 
seen  in  Fig.  8.  In  this  concept,  the  operator  has  to  turn 
dials  or  move  sliderbars  at  the  sides  of  a 
software-window  to  modify  the  display  area.  For 
desktop  computers  this  concept  is  familiar,  often  used 
and  nearly  standard.  However,  on  large  displays  it  was 
found  to  be  very  inconvenient.  The  reason  for  this  is  that 
reaching  for  the  navigation  control  means  to  move  a 
cursor  a  long  way  across  the  display  and  this  takes  a  lot 
of  time. 


Fig.  8:  Different  desktop  navigation  concepts, 
left  side:  MS  Windows  0  and  sliderbars; 
right  side:  Cosmoplayer0  and  virtual  cockpit 


Secondly,  concepts  for  “virtual  cockpits”  are  possible,  as 
shown  in  Fig.  8.  The  operator  moves/flies  through  the 


scene  by  controlling  a  throttle  and  a  virtual  control  stick 
shown  in  the  lower  middle  of  the  GUI.  This  technique  is 
often  used  for  desktop- VE  systems,  especially  for 
VRML-applications.  However,  for  larger  displays  and 
semi-immersive  VE-systems  it  has  been  found  to  be  not 
efficient  enough. 

For  reasonable  use  of  the  benefits  of  VE,  the  interaction 
has  to  be  more  intuitively.  The  concept  of  “grab-and- 
move ”  describes  a  first  step  in  this  direction.  In  this 
concept  the  operator  grabs  the  scene  and  slides  it  to  any 
direction.  First  trials  using  an  implementation  of  this 
concept  show  that  navigation  becomes  rather  easy  and 
fast. 

In  the  actual  experimental  setup  a  trackball  is  used  as 
main  interaction  device.  Missing  degrees  of  freedom  are 
simulated  by  additional  switches  on  the  device.  For 
future  studies  evaluations  of  other  devices  are  planned. 
One  of  those  will  be  a  virtual  laser  pointer  which  makes 
pointing,  selection  and  navigation  possible. 

6.2.2  Manipulation 

Manipulating  the  objects  of  the  virtual  environment  will 
use  the  same  devices  as  for  navigation.  A  switch  or 
control  bar  is  used  to  toggle  between  navigation  and 
manipulation  mode.  In  manipulation  mode  the  operator 
has  to  be  able  to  select  objects  and  choose  special 
control  operations. 

Main  control  operations  are: 

•  generation  of  new  objects, 

•  editing  attribute  values, 

•  erasing, 

•  placement, 

•  orientation, 

•  movement, 

•  editing  terrain  data. 

Generation,  editing  and  erasing  of  objects  can  easily  be 
implemented.  But  placement,  orientation,  movement  and 
editing  of  terrain  data  are  more  difficult,  because 
interactions  between  different  objects  of  the  scene  graph 
have  to  be  considered.  This  can  easily  lead  to  errors  in 
the  structure  of  the  scene  graph. 

For  correct  placement  and  orientation  of  objects  in  the 
three-dimensional  space  elevation  information  about  the 
base  point  is  necessary.  Moreover,  if  the  object  is  placed 
on  oblique  terrain,  it  has  to  be  oriented  correctly.  This 
information  can  only  be  derived  from  the  original 
geographic  databank  which  is  not  a  part  of  the  scene 
graph.  For  this  reason  data  exchange  with  the  original 
geographic  databank  has  to  be  possible. 

Movement  of  objects  is  even  more  critical.  It  describes 
the  process  of  detaching  the  object  out  of  the  scene 
graph  and  re-placing  it  at  another  position.  A  trivial 
implementation  would  be  to  erase  the  old  object  and 
generate  an  identical  new  object  at  the  new  place.  But 
intelligent  movement  algorithms  should  enable  a 
movement  of  the  object  while  simultaneously  fixating  it 
on  the  ground  level  {terrain  following). 

More  complex  edit  operations  of  the  terrain  databank  are 
very  difficult,  because  the  original  geographic  databank 
has  to  be  modified,  stored  and  re-converted  into  a  new 
scene  graph.  As  said  before,  this  has  to  happen  offline. 
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Consequently  the  actual  setup  of  the  Electronic 
Sandtable  does  not  support  these  operations. 

6.3  Cooperation 

The  concept  of  the  Electronic  Sandtable  has  been 
designed  to  enable  multiple  operator  working  in  the 
virtual  scene.  It  has  to  include  cooperation  concepts. 

In  contrast  to  full-immersive  VE,  in  semi-immersive  VE 
all  operators  are  present  at  the  same  location. 
Communication  and  inter-operator  interaction  happens 
the  natural  way.  Therefore  mainly  human-computer 
interaction  issues  have  to  be  analyzed.  These  main  issues 
and  problems  will  be  discussed  in  the  following. 

A  correct  perspective  visualization  of  the  computer¬ 
generated  picture  is  limited  to  only  one  single  operator 
due  to  technical  reasons.  The  technically  possible  frame 
rate  is  limited  to  a  maximum  of  144  Hz.  For  stereoscopic 
visualization  the  rate  is  cut  into  halves  (72  Hz).  An 
additional  operator  would  mean  to  cut  it  into  halves 
again  (36  Hz),  making  flickering  as  well  as  occurring  of 
hazardous  separation  of  single  pictures  likely  to  appear. 
In  a  word  it  can  be  summarized  that  with  today’s 
technology  only  calculation  and  stereoscopic 
presentation  for  a  single  operator  is  possible.  Further 
operators  will  perceive  a  perspective  error. 

One  way  of  dealing  with  the  problem  would  be  to  track 
more  than  one  operator  and  to  calculate  an  average 
position.  Average  position  does  not  necessary  mean  the 
arithmetic  mean,  it  could  be  a  more  complex  formula 
determined  by  subjective  ratings.  This  way  the 
perspective  error  might  be  minimized  and  might  not  be 
subjectively  perceivable. 

Apart  from  visualization,  cooperation  also  has  effects  on 
interaction  and  manipulation.  For  different  operators 
working  in  a  virtual  scene,  two  main  concepts  exist.  The 
concepts  are  drafted  in  Fig.  9. 

The  conference  concept  is  characterized  by  an  active 
presenter  and  passive  participants.  The  presenter  is  able 
to  navigate  and  manipulate,  whereas  the  participants  are 
following  the  presentation.  There  is  no  real  cooperation 
taking  place. 

On  the  other  hand,  in  the  workshop  concept ,  all 
participants  become  active.  However,  they  are  given 
different  rights  for  access.  At  the  beginning  of  each 
session,  an  operational  area  is  set  as  working  region. 
Afterwards  participants  are  able  to  work  in  this  region. 
This  concept  is  not  limited  to  human-computer 
interaction  but  has  to  include  cooperation  procedures 
between  session  participants  as  well. 


Fig.  9:  Conference  (left)  and  workshop  (right)  concept 

A  typical  example  is  handing  over  an  object.  Participant 
#1  selects  an  object  and  gets  access  rights  for  it.  For 
exact  placement  he  hands  it  over  to  participant  #2. 


Access  rights  move  over  to  participant  #2  as  well. 
Finally  participant  #2  drops  the  object  and  loses  access. 

These  concepts  require  the  introduction  of  new 
procedures  and  an  intense  research  in  this  area.  Because 
reality  is  still  to  be  modeled  natural  procedures  serve  as 
input  for  the  model.  But  just  modeling  reality  does  not 
consider  that  VE-systems  have  much  more  capabilities. 
Therefore  new  advanced  concepts  have  to  be 
formulated,  to  optimize  use  and  gain  benefits  of  the 
system. 

7  Conclusion  and  Future  Research 

In  this  paper  the  baseline  concept  of  using  semi- 
immersive  VE-technology  as  advanced  TSD  has  been 
described.  The  approach  has  been  shown  to  be 
promising  and  advantageous. 

It  has  been  emphasized  that  human  factors  and 
ergonomics  are  the  main  issues  for  reasonable 
VE-application.  Main  research  issues  were  found  to  be 
visualization,  interaction  and  cooperation.  But  these 
topics  cannot  be  analyzed  separately,  because 
interactions  between  them  exist.  In  this  paper  some 
research  issues  were  introduced  and  results  of  ongoing 
research  studies  in  the  area  of  visualization  were 
presented. 

But  even  if  in  future  the  system  works  as  it  is  supposed 
to  be,  one  question  to  be  answered  still  remains:  The 
question  for  quantification  of  the  profit  and  gain  of  using 
VE-systems.  The  key  criteria  for  answering  this  question 
will  be  performance  of  the  human-VE  system. 

For  this  reason  human  performance  metrics  will  have  to 
be  introduced,  formulated  and  analyzed.  They  should  be 
as  fundamental  as  possible,  but  still  take  into  account  the 
characteristics  of  the  application. 

Jointly  with  other  basic  research  studies  they  will  be  the 
key  issues  of  future  research  in  this  area. 
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Summary:  This  paper  outlines  ongoing  research 
into  the  application  of  shared  large  screen 
displays  (LSDs)  for  visualising  the  battlespace. 
This  research  was  funded  by  the  MOD’S 
Corporate  Research  Programme  1999-2000 
(TG5:  Human  Sciences  and  Synthetic 

Environments).  Completeness  and  accuracy  of 
information  has  the  potential  to  enhance 
awareness  of  the  situation  and  increases  the 
probability  of  better  decision  making  by  the 
command  team.  Thus  LSDs  may  lead  to 
enhanced  team  situation  awareness,  and  it  is 
proposed  that  this  in  turn  should  lead  to  more 
effective  decision  making.  The  paper  discusses 
panoramic  displays  in  relation  to  the  potential 
benefits  that  would  arise  when  military 
command  teams  use  them.  Understanding  of  the 
command  team’s  information  requirements  is 
crucial  to  the  operational  advantage  gained  with 
LSDs.  This  paper  also  reports  the  outcome  of  a 
user  requirements  capture  exercise,  where  users 
were  required  to  consider  their  potential 
information  display  requirements  for  an  LSD  to 
be  used  in  2010. 

1  Introduction 

The  panoramic  display  is  essentially  a  shared 
visual  environment  (with  a  display  area  of  larger 
than  10  square  feet)  providing  the  operational 
teams  with  information  on  the  status  of  assets. 
Applications  of  such  displays  can  be  as  diverse 
as  operational  pictures  presented  to  battlefield 
commanders,  to  trainers  viewing  trainees  in 
multi- ship  synthetic  environments. 

Large  screen  displays  (LSDs)  as  a  concept  are 
not  new.  In  the  Second  World  War  RAF 
controllers  viewed  a  large  desktop  display  of  the 
air  picture.  There  are  also  civil  applications  in 
safety  critical  environments  such  as  nuclear 
power  stations.  However,  recent  advances  in 
computing  power  have  enabled  digitisation  of 
many  aspects  of  the  military  environment.  These 
include  higher  level  requirements  such  as  the 
presentation  of  the  joint  operational  picture  and 
specific  vehicle-based  concepts  providing 
battlefield  situational  awareness  and  command. 


Nonetheless,  the  development  of  technology  for 
large  screen  displays  leaves  the  questions 
begging  as  to  where  the  real  military  benefits  are 
to  be  gained?  In  particular,  in  team 
environments,  how  can  the  LSD  help  improve 
operational  decision  making?  Other  important 
questions  remain  unanswered:  how  does  the 
design  of  the  LSD  impact  on  Human  Computer 
Interaction  (HCI)  and  how  can  this  be 
measured?  The  issues  underlying  these 
questions  provided  the  framework  for  the  project 
reported  here.  The  objectives  of  the  research  are: 

•  To  reach  an  understanding  of  the  potential 
performance  benefits  of  LSDs. 

•  To  conduct  empirical  evaluations  to  develop 
optimal  HCI  techniques  and  formats  for 
interfacing. 

•  To  understand,  at  a  conceptual  level, 
changes  to  and  benefits  of  team  decision 
making  with  shared  information  space. 

1.1  Background 

Current  and  future  military  operations  will 
depend  chiefly  on  the  C4ISR  (Command, 
Control,  Communications,  Computing, 
Intelligence,  Surveillance  and  Reconnaissance) 
framework.  The  force  most  likely  to  prevail  in 
any  operation  is  the  one  that  accumulates 
intelligence  faster  and  more  accurately  than  its 
opposition.  This  involves  the  collection, 
dissemination,  processing  and  interpretation  of 
large  volumes  of  information. 

The  current  operational  environment  has 
resulted  in: 

•  Equipment  and  missions  becoming  more 
complex. 

•  Risk  of  data  overload. 

•  Team  decision  support. 

•  Tri-service  and  multinational  missions  and 
introduction  of  joint  structure. 

To  support  missions  that  are  becoming 
increasingly  complex,  and  thus  requiring 
commanders/operations  room  personnel  to  act  as 
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information  assimilators  and  overseers, 
information  displays  are  being  revolutionised. 
Further,  as  the  likelihood  of  future  missions 
being  joint  and/or  multinational  has  increased, 
the  joint  tactical  picture  is  of  primary 
importance  to  mission  controllers.  Providing 
timely,  accurate  and  relevant  information  means 
much  of  it  may  need  to  be  filtered  out  to  protect 
a  commander  from  data  deluge  and  facilitate  his 
decision-making  process. 

Technologies  such  as  liquid  crystal  displays 
(LCDs),  back  projection  systems,  large  plasma 
displays,  roll-up  portable  displays  and  ‘picture 
window’  flat  screens  are  already  being  used  in 
service  or  are  in  development.  These  will  enable 
high  resolution,  integrated  information  displays 
to  be  presented  on  wide  angle,  panoramic 
displays  simultaneously  to  a  number  of 
personnel.  Moreover,  the  development  of 
interactive  technologies  (such  as  shared  virtual 
environments,  smartboards  and  shared 
whiteboards)  will  enable  operators  to  interact 
directly  with  the  LSD  using  novel  inputs  (voice, 
eye,  and  touchscreen)  over  large  distances. 

Hence  distributed  command  teams  will  become 
feasible.  HCI  issues  are  pertinent  to  such 
capabilities:  who  has  the  authority  to  interact 
with  and  change  data  is  fundamental.  Potential 
applications  are  vast  and  diverse.  The  command 
team  could  be  in  airborne  command  centres,  on 
ships,  or  unmanned  air  vehicles  (UAV) 
command  stations.  Battlefield  displays  could  be 
presented  with  data  fused  information  on  one 
single,  splayed,  panoramic  screen,  which  all 
individual(s)  could  view;  potentially  facilitating 
shared  mental  models,  situation  assessment  and 
decision  making.  This  project  focused  on  an 
LSD  application  relevant  to  the  three  services 
and  joint  operations,  that  is  the  presentation  of 
battlefield  information  to  the  command  team.  It 
is  envisaged  that  this  particular  use  of  LSDs  will 
have  considerable  relevance  to  current 
procurements  and  provide  generic  guidelines  for 
other  applications. 

1.2  Guidelines  for  Panoramic  Displays 

Little  is  understood  of  how  LSDs  improve 
operator  performance  and  knowledge  of  the 
related  human  factors  issues  is  minimal.  For 
example,  most  of  the  existing  HCI  research  is 
related  to  standard  individual  workstations. 
There  are  few  recommendations  for  how 
interfaces  on  LSDs  should  be  designed,  and  how 
LSDs  should  be  integrated  within  the  user’s 
operational  environment  and  task  structure. 
McLeod  [1]  recommends  that  LSDs  are 
potentially  suitable  to  support  team  working 


when  one  or  more  of  the  following  conditions 
are  met: 

•  Operators  require  concurrent  use  of 
information/displays 

•  Operators  have  shared  tasks 

•  There  are  non-conflicting  task  needs 

•  Operators  have  common  information  needs 

•  Feedback  is  required  to  be  given  to  whole 
teams 

•  Operator  tasks  require  a  common  frame  of 
reference  (i.e.  an  indication  of  where  an 
operator  is  within  the  big  picture,  to  supply 
overall  context,  and/or  help  prioritise  an 
individual’s  operation) 

•  Operator  tasks  require  high-level 
summary/overview  information. 

Further  definitions  of  the  HCI  requirements  for 
interacting  with  LSDs  are  scarce.  There  are  three 
main  categories  of  human  factors  design 
requirements  relating  to  physical  properties, 
information  display  requirements  and  workspace 
requirements  that  must  be  met  to  optimise 
performance  when  using  LSDs: 

•  Physical  display  requirements  include  full 
colour  capability,  sufficient  viewing  angle, 
high  resolution  and  uniform  light 
distribution. 

•  Information  display  requirements  include 
the  need  for  a  lack  of  clutter  on  the  display 
and  the  need  for  critical  information  not  to 
be  altered  or  deleted  accidentally. 

•  Workspace  requirements  mean  that 
panoramic  displays  should  be  functionally 
consistent  with  the  physical  room  layout 
and  integrated  with  command  post  positions 
and  roles.  Operators  must  also  be  familiar 
with  using  LSDs. 

Finally,  how  LSDs  are  applied  to  the  team 
environment  is  critical.  Stubler  and  O’Hara  [2] 
generated  several  guidelines  to  be  applied  to  the 
use  of  shared  LSDs  in  order  to  enhance  team 
performance,  which  were  grouped  into  the 
following  four  sets  of  functions: 

•  Provide  a  status  overview :  to  support  high 
level  decision  making  and  enhance  team 
memory,  where  operators  are  performing 
multiple  concurrent  tasks. 

•  Direct  staff  to  additional  information :  by 
posting  critical  information  to  update 
individuals’  knowledge  states. 

•  Aid  co-ordination  of  team  activities :  to 
facilitate  team  identity  and  clearly 
communicate  common  tasks,  where 
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multiple  tasks  and  proceduralised 
performance  are  present. 

•  Support  collaboration  among  team 
members :  by  acting  as  a  focal  point  for  team 
discussions. 

It  is  thus  possible  that  the  use  of  panoramic 
displays  may  enhance  team  performance  in  a 
military  Command  and  Control  (C2) 
environment.  This  leads  to  the  important  issue  of 
how  panoramic  displays  will  impact  on  team 
processes  within  the  command  team. 
Understanding  team  processes  and  models  is 
crucial  in  understanding  and  exploiting  the 
effect  that  panoramic  displays  may  have  on  team 
performance  and  processes  in  a  military 
command  setting. 

1.3  Panoramic  Displays  in  the  command 
centre 

LSDs  have  been  used  for  several  years  in  a 
range  of  civil  command  centre  applications, 
such  as  power  industries  (nuclear  and  electricity) 
and  transport  systems  (rail  and  air),  where 
monitoring  of  the  status  of  safety-critical 
systems  is  essential.  These  displays  are  used 
mainly  in  normal  operations,  but  need  to  support 
decision  making  in  emergencies.  As  the 
information  portrayed  is  often  complex,  these 
LSDs  enable  the  overseer  to  trace  an  event  back 
to  its  source.  In  contrast,  the  military  LSDs  will 
need  to  maintain  rapidly  changing,  unpredictable 
and  conflicting  information  with  safety  still 
being  a  concern,  but  mission  success  being  the 
overriding  factor.  It  is  how  LSDs  will  impact  on 
team  performance  and  team  processes  (e.g. 
intra-team  communication)  in  a  military 
environment  that  is  little  understood  [3]. 

The  use  of  LSDs  is  likely  to  have  an  affect  on 
the  following  main  aspects  of  team  processes: 
communications,  leadership,  situation 
awareness,  shared  mental  models,  decision 
making,  workload  management,  mission 
analysis,  planning  and  adaptability,  and 
teamwork. 

Recent  studies  that  concentrated  on  realistic 
scenarios  in  a  military  setting,  showed  that 
teams  do  tend  to  perform  better  when  working 
with  shared  LSDs.  For  example,  Hiniker  and 
Entin  found  that  shared  battle  graphics  presented 
on  a  large  overhead  screen  seemed  to  increase 
the  effectiveness  of  teams  by  providing  more 
accurate  information  concerning  a  wider 
overview  of  the  situation.  In  a  subsequent 
experiment  [4],  participants  reported  that 
understanding  crisis  situation  and 


communicating  knowledge  of  the  situation  was 
easier  and  better  with  the  use  of  LSDs. 

Hiniker  [5]  has  discussed  in  depth  the  use  of  a 
Common  Operational  Picture  (COP).  The  COP 
is  an  annotated  electronic  map  of  the  battlefield, 
showing  the  location  of  own  and  enemy  forces. 
The  COP  updates  rapidly,  allowing  commanders 
to  visualise  the  situation  and  hypothesise 
possible  courses  of  action.  Wentz  [6]  reported 
the  use  of  the  COP  in  Bosnia  in  1998.  It  was 
found  to  be  useful,  but  better  integration 
between  pictures  was  felt  to  be  necessary. 

Nonetheless  despite  the  expected  benefits  of 
LSDs,  there  are  few  studies  identifying  user 
requirements  or  describing  empirical  evidence  to 
support  the  anticipated  benefits.  The  intention  of 
this  paper  is  to  outline  the  knowledge  elicitation 
phase  of  this  research  project  that  feeds  into  the 
prototype  battlefield  LSD,  which  is  being 
developed  as  part  of  this  project. 

In  parallel  with  this  knowledge  elicitation  phase 
a  communication  analysis  of  a  simulated 
command  team  exercise  was  carried  out.  This  is 
reported  in  detail  in  [7].  As  Table  1  illustrates, 
the  most  common  type  of  communication  was 
providing  situation  updates  (18.9%  of  the  total 
number  of  communications),  followed  by 
requests  for  information  (15.3%)  and  providing 
information  on  request  (14.7%).  It  was 
important  that  these  requirements  were  met  in 
the  design  phase. 

2  Knowledge  elicitation 

The  objective  of  this  phase  of  the  project  was  to 
understand  how  LSDs  could  provide  decision 
support  from  the  command  team  perspective.  In 
order  to  meet  this  objective  two  command  teams 
were  selected  as  potential  users  of  LSD 
technology  and  knowledge  elicitation  techniques 
were  utilised  to  identify  their  requirements.  This 
involves: 

•  Understanding  the  users’  HCI  requirements 

•  Focus  groups  with  two  command  teams 
were  conducted  to  identify  command 
team  information  needs 

•  Encapsulating  their  needs 

•  Trials  to  assess  cognitive  benefits  will 
follow  on  from  the  identification  phase 
assessing  command  performance. 

As  a  consequence  of  the  increasing  emphasis  on 
joint  operations,  the  first  end-user  team  was 
drawn  from  Operations  Team  Staff  in  the  UK 
Permanent  Joint  HQ  (PJHQ)  representing  joint 
operational  and  strategic  crisis  management.  The 
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second  command  team  represented  the  single¬ 
service  viewpoint,  in  this  case  staff  from  the 
Combat  Operations  team  at  the  UK  Combined 
Air  Operations  Centre  (CAOC). 

In  order  to  identify  key  applications  for  LSDs, 
two  methods  were  utilised  with  Subject  Matter 
Experts  (SMEs).  These  were  cognitive 
walkthroughs  and  focus  groups.  Focus  groups 
are  a  direct  and  qualitative  research  approach, 
often  in  the  form  of  an  interview  conducted  by  a 
trained  moderator  among  a  small  group  of 
experienced  respondents  in  a  natural  manner  [7]. 
This  is  all  done  in  order  to  gain  an  insight  into 
the  group  subject,  through  instilling  a  relaxed, 
informal  atmosphere  that  encourages 
spontaneous  comments. 

Cognitive  walkthrough  is  a  technique  based  on 
cognitive  theory  for  evaluating  the  design  of  a 
user  interface,  it  attempts  to  elicit  information  on 
how  well  the  interface  supports  ‘exploratory 
learning’.  Cognitive  walkthroughs  aim  to 
evaluate  four  aspects;  the  user’s  goal,  the 
accessibility  of  the  correct  control,  the  quality  of 
the  match  between  the  control’s  label  and  the 
goal,  and  the  feedback  provided  after  the  control 
is  acted  on.  This  method  is  often  used  in  the 
early  stages  of  design,  before  empirical  user 
testing  is  possible.  The  method  involves  several 
approaches  ranging  from  heavily  structured 
questionnaires  to  more  informal  but  structured 
group  sessions.  Such  groups  are  normally 
composed  of  the  interface  designer  and  a  group 
of  his  or  her  peers.  One  of  the  group  takes  the 
role  of  ‘scribe’  (recording  the  results)  and 
another  takes  the  role  of  ‘facilitator’  (keeping 
the  evaluation  moving)  [7]. 

As  the  aim  of  the  PJHQ  and  CAOC  visits  was  to 
capture  information  on  military  command 
teams’  concepts  of  LSDs,  capturing  these 
concepts  required  a  flexible  method.  After 
considering  the  qualities  of  both  focus  groups 
and  cognitive  walkthrough,  it  was  decided  that 
the  exercise  used  for  both  PJHQ  and  CAOC 
would  be  an  informally  structured  group  session. 
This  was  aimed  at  evoking  a  relaxed  atmosphere 
that  would  capture  both  answers  to  set  questions 
and  spontaneous  comments  [7]. 

During  each  exercise,  the  command  teams  were 
asked  to  identify  potential  uses  of  LSDs  in  2010 
and  then  were  asked  to  prioritise  information 
that  could  be  displayed  to  the  team.  Following 
on  from  this  understanding  of  how  LSD 
information  could  support  team  situational 
awareness,  the  SMEs  were  asked  to  design  a 
prototype  LSD.  Finally,  in  the  case  of  the  single 
service  staff,  there  was  a  walkthrough  of  a 


possible  scenario  to  confirm  the  layout  and 
utility  of  the  LSD. 

3  Results 

3.1  Single  service  LSD 

This  exercise  was  designed  to  capture 
operational  staff  views  on  how  LSDs  could 
support  the  command  team  in  air  combat 
operations.  It  was  also  aimed  at  identifying  how 
critical  information  should  be  displayed  to  and 
interfaced  with  by  the  team.  While  it  was  not  the 
intention  to  derive  particular  screen  layouts,  a 
default  screen  layout  was  identified  as  part  of 
the  group  activity  (Figure  1). 

As  a  result  of  these  exercises,  the  following  two 
applications  of  LSDs  were  evident:  briefing  and 
team  situational  awareness.  In  particular,  the 
LSD  would  benefit  staff  shift  changes  and 
maintain  team  awareness  of  events  that  they 
were  not  party  to  at  their  own  consoles.  The 
commander  commented  that  the  LSD  made  a 
good  focus  for  team  attention  during  briefings. 

Displaying  weather  information  was  an 
important  LSD  requirement.  This  was  viewed  as 
an  important  factor  for  hand-over  briefings, 
predictions  and  continual  reference  during 
operations.  As  with  all  the  information 
displayed,  it  was  suggested  that  such  a  display 
should  be  totally  flexible  and  able  to  be  applied 
to  any  situation. 

Unlike  the  joint  team  exercises,  the  single 
service  operational  team  did  not  view  news 
feeds  and  video  conferencing  as  essential.  The 
team  suggested  that  the  prime  information 
source  for  the  LSD  should  be  exactly  what  the 
Chief  of  Combat  Operations  (CCO)  desires  on 
the  screen  at  any  particular  time.  Interaction 
should  be  mainly  through  the  CCO,  although 
others  should  be  able  to  take  over  and 
manipulate  the  LSD  as  necessary,  e.g.  when 
briefing.  It  is  useful  within  a  scenario  for  each 
member  of  the  Operations  team  to  be  able  to 
brief  in  turn,  using  the  LSD  where  necessary  to 
update  all  team  members’  situational  awareness. 
This  raised  the  issue  of  live  bulletin  boards.  The 
team  suggested  that  such  a  concept  would  prove 
to  be  a  useful  tool.  The  idea  behind  a  ‘bulletin 
board’  would  be  a  common  area  where  any  one 
of  the  Combat  Operations  team  could  project 
information  onto  the  LSD,  thereby  attracting  the 
shared  attention  of  the  team  to  important 
information.  In  addition  it  was  suggested  that  the 
screen  should  have  a  ‘zoom’  function  that  would 
allow  any  relevant  information  to  be  selected 
from  the  screen  and  be  re-sized  as  required. 
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As  a  final  product  of  the  exercise,  a  default 
screen  layout  for  an  LSD  with  suggested 
functionality  was  proposed  (Figure  1).  It  was  felt 
essential  that  all  windows  were  moveable  and 
re- sizeable.  The  interface  should  be  controlled 
by  the  CCO  (except  for  the  bulletin  window). 
All  team  members  should  have  access  to  a 
remote  mouse  for  giving  briefings,  enabling 
them  to  highlight/manipulate  windows  of 
interest.  Further,  the  team  should  be  able  to 
record  all  actions,  so  they  could  review  events 
and  save  useful  configurations  for  later  use. 

3.2  Joint  LSD 

A  similar  exercise  and  discussions  were 
conducted  with  the  joint  team  operating  at 
strategic/operational  level.  The  following 
summarises  user  requirements: 

•  The  interface  should  utilise  current 
computer  formats,  for  example  the  use  of 
windows,  drop  down  menus  and  icons. 

•  All  screen  formats  should  be  reconfigurable. 

•  Users  should  have  the  ability  to  move 
between  levels  of  information  at  the  click  of 
a  button;  for  example  view  a  map  at 
different  resolutions. 

As  the  joint  operations  team  controls  a  number 
of  operations  simultaneously,  the  LSD  must 
have  the  ability  to  display  the  information  for  a 
number  of  operations  at  the  same  time.  The 
background  of  each  section  of  the  display  was 
the  respective  operational  picture.  Extra 
information  required  for  the  task  from  a  range  of 
sources  including  video,  imagery,  newsfeeds, 
video  conferencing,  briefings,  etc.  can  also  be 
displayed  (see  Figure  2). 

Furthermore,  when  required,  it  may  be  necessary 
to  promote  one  of  these  operational  areas  to  fill 
the  entire  LSD  with  the  remaining  operations 
being  minimised.  It  was  also  important  that  the 
LSD  reflected  the  control  room  layout  and 
operating  functions. 

In  addition  to  team  situational  awareness,  the 
LSD  would  be  used  to  brief  important  visitors. 
The  joint  team  was  often  in  the  position  where 
they  are  required  to  display  information 
centrally.  Therefore  there  is  a  requirement  for 
the  whole  screen  to  become  a  dedicated  briefing 
facility. 

It  was  clear  that  from  the  joint  perspective  there 
was  a  strong  user  requirement  for  an  LSD  and 
support  for  a  flexible,  versatile  system. 


4  Generic  HCI  guidelines  for  LSDs 

As  a  result  of  the  exercises  investigating  optimal 
LSD  format,  involving  joint  and  single  service 
staff,  three  general  categories  of 
recommendations  for  display  design  have 
become  clear.  These  fall  under  the  headings  of: 
display  configuration,  interaction  procedures 
and  user  group  considerations. 

4.1  Display  configuration 

A  large  screen  display  should: 

•  Be  flexible  and  re-configurable  to  the  needs 
of  different  user  teams  and  their 
composition. 

•  Have  the  ability  to  move,  re-size  and 
overlay  any  information  sources  onto  and 
over  any  area  of  the  LSD. 

•  Have  the  ability  to  zoom  in  or  out  of  any 
area  on  the  main  display. 

•  Display  information  at  its  highest  level; 
plans  and  workings  are  not  recommended 
for  display  (unless  the  tasks  of  the  team 
demand  it). 

•  Make  the  best  use  of  the  available  space  in 
any  section  of  the  display. 

4.2  Interaction  procedures 

•  LSDs  must  be  simple  and  straightforward  to 
operate. 

•  LSDs  must  conform  to  population 
stereotypes  (i.e.  user  expectations  about 
how  the  system  will  work). 

•  Access  to  and  exit  from  any  LSD  systems 
should  be  via  one  procedure  (e.g.  one  click 
of  a  mouse  button). 

•  There  needs  to  be  a  process  whereby  useful 
layouts  and/or  settings  can  be  saved. 

•  Movement  between  different  levels  of 
information  should  be  at  the  click  of  a 
button. 

•  The  LSD  should  provide  an  ‘on-line’  help 
procedure. 

•  The  LSD  should  have  a  system  that  runs 
hand-in-hand  with  a  time-line  of  significant 
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events.  Therefore  providing  traceability  of 
all  decisions. 

4.2  User  group  considerations 

•  LSDs  must  be  designed  around  how  the 
user  team  works,  rather  than  dictating  how 
the  user  team  works. 

•  LSDs  must  support  communication.  This 
can  be  achieved  by  matching  the  needs  of 
the  team  with  the  information  that  is 
displayed  on  the  LSD  to  support  intra-team 
communication. 

•  Interaction  with  the  LSD  should  be 
primarily  through  the  CCO,  however  any 
member  of  the  command  team  should  have 
the  ability  to  manipulate  the  LSD  for 
briefing  purposes  via  a  single,  communal 
remote  mouse. 

•  It  is  essential  not  to  design  out  the  role  of 
the  individual.  LSDs  should  be  used  to 
increase  situational  awareness  for  all  team 
members  and  allow  for  individuals  to  carry 
out  individual  tasks  independently. 

•  There  should  be  the  provision  of  two  types 
of  LSD,  a  fixed  version  and  a  deployable 
version.  The  deployable  LSD  version 
should  be  portable,  robust  and  reliable.  Both 
versions  should  be  functionally  similar  to 
avoid  the  need  for  additional  training. 

5  Conclusions 

The  changing  nature  of  world  security  has 
dictated  the  requirement  for  flexible  and 
deployable  forces.  Some  of  the  impacts  of  the 
security  environment  and  technology  progress 
on  military  operations  have  been  reductions  in 
crew  levels,  interest  in  remote  control  of 
systems,  e.g.  Unmanned  Air  Vehicles  (UAVs), 
and  the  need  for  complex  command  and  control 
structures.  The  philosophy  of  developing  an 
interface  that  allows  team  members  to  develop 
an  accurate  mental  model  of  the  location, 
activity  and  performance  of  assets  may  be  a  key 
component  in  minimising  the  risk  of  operator 
error  due  to  an  incomplete/incorrect  awareness 
of  the  operational  situation.  The  introduction  of 
LSDs  will  have  the  potential  military  benefit  of 
allowing  the  commander  and  command  team  to 
share  information  space,  potentially  enhancing 
situational  awareness  and  team  decision  making. 


methods  to  assess  team  performance  benefits. 
As  a  result  of  this  research,  there  is  a  greater 
understanding  of  how  operational  pictures  are 
currently  used  and  how  future  LSDs  can  be 
designed  to  optimise  command  team 
performance.  This  report  has  identified  LSD 
HCI  requirements  for  operational  tasks  that  can 
be  used  in  future  designs. 
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Communication  type 

Number 

Percentage  of  total 
communication 

Providing  situation  updates 

240 

18.9 

Requesting  information 

194 

15.3 

Providing  information  on  request 

186 

14.7 

Providing  information  without  being  asked 

104 

8.2 

Commands/orders 

102 

8.1 

Questions/enquiries 

91 

7.2 

Seeking  clarification 

74 

5.8 

Acknowledge/verify/confirm 

47 

3.7 

Observations 

40 

3.2 

Providing  assistance/backup 

38 

3.0 

Guidance/suggestions 

34 

2.7 

Providing  clarification 

32 

2.5 

Requesting  backup/assistance 

21 

1.7 

Repeating  information 

20 

1.6 

Stating  priorities 

17 

1.3 

Social/other 

13 

1.0 

Giving  feedback 

5 

0.4 

Prompts 

4 

0.3 

Error  correction 

3 

0.2 

Agreement 

2 

0.2 

Table  1 :  Communication  types  in  frequency  order 


All  windows  around 
the  exterior  can  be 
minimised  as  icons, 
capable  of  being 
moved,  resized  and 
overlaid 


Whiteboard  area  -  open  channel 
for  all  to  enter  notes  or  to  feed 
information  from  personal  /j 

workstation 


Datalink 

picture 


White  board 


Military 
Alert  Status 


1  Data  sources 

Video 

- _ - __ 

L 

i - 

Imagery 

footage 

r 

Weather 


CCTV/video 

feed 


Briefing  area-  re-sizeable  and  able  to  dominate  the  whole 
screen;  when  in  this  mode  operational  picture/other  screens 
run  minimised  or  as  an  icon  in  one  comer.  Need  to  be  able 
to  pull  up  individual  screens  and  overlay  on  top  of  enlarged 
briefing  screen. 


Common  operational  picture 


Rules  of 
engagement 


Exclusion  zone 


Threat  information 


Tasking 
orders _ 


24  hr  coverage 


Permanent,  re-sizeable 
windows  containing 
dymamic  information; 
other  windows  may  be 
overlaid 


Platform 

imagery 


Battle  assessment 


Commander’s 

threat 

assessment 


Time/events 

log 


Data  windows 
layered/tiled  sub¬ 
windows  so  that 
each  can  be 
overlaid  and  the 
area  of  interest 
can  be  bought  to 
the  top 


Runs  as  an  icon 

so  staff  can 

maintain 

historical 

situational 

awareness 


Mission  Specific  Information  - 

layered/tiled  sub-windows  so  that 
each  can  be  overlaid  and  the  area 
of  interest  can  be  brought  to  the 
top 


Figure  1:  Generic  single  service  LSD  layout 
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♦(The  operational  boxes  OP  1,  2,  3  &  4  represent  which  operation  each  quadrant  will  be  displaying  and 
will  not  appear  on  the  LSD.) 

Figure  2:  Joint  services  LSD  screen  layout  (for  four  operations). 
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Summary 

This  paper  addresses  new  interface  concepts  for 
information  visualisation  and  manipulation  in 
Command  and  Control.  These  concepts  focus  on 
the  use  of  multiple  views  on  the  tactical  situation  to 
enhance  situational  awareness  and  to  improve 
situation  assessment.  Topics  covered  include  the 
application  of  3D  perspective  and  stereoscopic 
displays. 

INTRODUCTION 

With  command  and  control  operations  becoming 
increasingly  more  complex,  there  is  a  growing  need 
for  research  on  and  development  of  effective 
command  support  tools.  Probably  due  to  the  strong 
emphasis  on  information  and  technology,  but 
certainly  due  to  shortcomings  in  our  present 
understanding  of  command  and  control  tasks,  and 
the  human  capabilities  they  call  on,  many  of 
today’s  command  and  control  systems  are  more 
designed  from  an  information  system  perspective 


than  from  a  command  support  perspective. 

To  get  a  better  understanding  of  tasks  and  needs  for 
support,  the  analysis  of  tasks  and  the  identification 
of  critical  performance  shaping  factors  in  the  CIC 
have  been  subject  of  several  studies  for  the 
RNethN.  What  can  be  learned  from  these  studies  is 
the  need  for  better  tools  to  support  situational 
awareness  and  situation  assessment. 

Easy  access  to  information  needed  for  these 
processes  of  awareness  and  assessment  strongly 
depends  on  how  the  environment,  situation,  plans 
and  system  states  are  visualised  and  the  way  these 
views  can  be  manipulated.  Started  as  monochrome 
representations  of  raw  radar  and  sonar  data,  the 
present-day  tactical  picture  has  developed  into  full- 
colour,  computer-generated  graphical  representations 
of  tracks  (figure  1),  serving  as  a  basis  for  the 
presentation  and  flexible  manipulation  of  different 
kinds  of  tactical  and  geographical  information. 
Within  this  scope  some  new  interface  concepts  for 
information  visualisation  and  manipulation  will  be 


Figure  1:  Evolution  from  monochrome  vector  display  technology  to  present-day,  full-colour  raster-scan  or  flat- 
panel  display  technology.  As  a  result  of  these  developments,  more  possibilities  for  the  presentation  and 
manipulation  of  tactical  data  are  offered. 


Paper  presented  at  the  RTO  HFM  Symposium  on  “Usability  of  Information  in  Battle  Management 
Operations",  held  in  Oslo,  Norway,  10-13  April  2000,  and  published  in  RTO  MP-57. 
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discussed.  These  include  the  use  of  multiple 
windows  and  (a.o.  ‘3D’)  views  on  the  tactical 
situation  to  enhance  situation  awareness,  and  the 
application  of  transparent  information  layers  for  an 
integrated  presentation  format.  Results  of 
experiments  with  the  concept  of  multiple  views 
indicate  that  higher  information  transfer  rates  could 
be  obtained,  together  with  a  large  decrease  of 
required  user  interaction  with  the  system.  Finally, 
as  an  extension  of  the  principles  described  here,  the 
concept  of  tactical  objects  is  introduced.  This 
concept  enables  the  user  to  modify  and  reconfigure 
the  tactical  workstation  for  effective  supervision  of 
and  a  rapid  response  to  the  tactical  situation  at 
hand. 

Although  the  proposed  concepts  certainly  need 
further  testing  and  development  for  successful 
application,  it  will  be  concluded  that  they  offer  a 
promising  perspective  for  the  development  of 
future  information  and  task  management  tools. 


MULTIPLE  WINDOWS 

In  current  command  information  systems  and 
combat  direction  systems,  information  on  the 
environment  is  mosdy  presented  in  a  single,  two- 
dimensional  ‘bird’s-eye  view’,  commonly  denoted 


as  the  tactical  picture.  Inherent  to  this  single-view 
presentation  type,  is  the  constant  need  to  zoom  in  or 
out  and  to  pan  horizontally  or  vertically  to  keep  the 
right  focus  on  the  tactical  situation.  Larger  scale  or 
longer  distance  views  have  to  provide  the  necessary 
overview,  smaller  scale  or  short  distance  views  the 
detailed  information.  In  many  cases  it  is  hard  to  get 
both  objectives  combined  in  one  unique  setting  of 
the  tactical  display.  Larger  scale  or  longer  range 
settings  often  produce  clutter  or  cannot  offer 
information  in  enough  detail.  Smaller  scale  or 
shorter  range  settings  required  for  the  detail  often 
do  not  offer  enough  context  for  a  more  global  view 
of  the  tactical  situation,  and  entail  the  danger  to  be 
‘taken  by  surprise’  where  relevant  information 
stays  out  of  sight.  Switching  between  different 
display  settings  takes  time.  Not  merely  because  of 
operator  actions  needed,  or  because  of  system 
response  times,  but  above  all  because  of  the 
inevitable  re-orientation  on  what  is  displayed  when 
settings  are  changed.  A  situation  in  which  sudden 
transitions  can  cause,  as  indicated  by  Woods  [1],  a 
loss  of  visual  momentum.  As  a  consequence,  when 
under  time  pressure,  operators  may  become 
reluctant  to  change  settings,  even  if  the  way 
information  is  displayed  does  not  suite  the  current 
situation. 

Within  this  scope  a  study  was  performed  on 


Figure  2:  Example  for  the  application  of  the  multiple-window  principle  with  operator  selected  parts  of  the 
tactical  picture  presented  in  separate  windows  for  close  monitoring.  Every  window  has  its  own  tools  for  filtering, 
display  settings,  positioning  and  sizing  of  windows.  Separate  windows  can  be  temporarily  ‘closed’  through 
minimising  to  icons  presented  on  the  overview  display. 
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possible  techniques  to  enhance  the  presentation  of 
tactical  situations  [2].  The  prime  research  goal  for 
this  study  was  the  support  of  warfare  officers  in 
their  situation  awareness  through  parallel 
presentation  of  multiple  windows  on  the  tactical 
situation  at  hand,  and  flexibility  in  display 
configurations  to  support  a  situation  dependent, 
optimised  presentation. 

The  technique  of  multiple  windows  enables  the 
operator  to  monitor  different  parts  of  the  tactical 
environment  in  a  parallel  way  and  at  different 
levels  of  detail  (see  figure  2). 

In  the  experimental  set-up  as  tested,  operators  had  a 
tool  at  their  disposal  to  select  parts  of  the  tactical 
picture  (the  overview  window)  for  presentation  in 
separate,  additional  windows.  These  windows  have 
their  own  tools  for  filtering,  display  settings, 
positioning  and  sizing  of  windows.  The  windows 
can  be  temporarily  'closed’  through  minimising  to 
icons  presented  on  the  overview  display. 

It  was  expected  that  the  use  of  multiple  windows 
would  have  a  positive  effect  on  the  alertness  of 
operators  with  respect  to  changes  in  the 
environment.  To  test  this  expectation,  eight  warfare 
officers  took  part  in  an  experiment  in  which 
performance  in  terms  of  information  transferred 
and  effort  needed  was  measured  for  both  a 
conventional  ‘single  window’  set-up  (SW)  and  an 
experimental  ‘multiple  window’  set-up  (MW). 
Subjects  were  asked  for  two  different  Anti  Air 
Warfare  and  Anti  Submarine  Warfare  scenarios, 
while  monitoring  the  tactical  situation,  to  detect  and 
indicate  as  fast  as  possible  changes  in  track 
behaviour,  like  course  and  speed,  and  other  state 
variables. 

Changes  to  be  detected  in  these  scenarios  could  be 
short-lived,  representing  the  upper  part  of  the 
transition  frequency  domain,  or  more  lasting  in 
different  degrees,  representing  the  lower  part  of  the 
transition  frequency  domain.  For  the  different  parts 
in  this  frequency  domain  information  transfer  was 
measured  in  terms  of  detected  changes  relative  to 
changes  present  in  the  scenario.  For  the  analysis 
these  changes  could  be  weighed  according  to 
tactical  relevance. 

In  figure  3  the  information  transfer  function  of 
tactically  very  relevant  transitions  (’high-priority 
targets')  is  presented  for  the  multiple  and  the  single 
window  interface. 

The  results  show  that,  given  enough  time  (low 
transition  frequency),  nearly  all  ‘high  priority’ 
transitions  are  detected  and  the  information 
transferred  approaches  the  maximum  value  of  1. 
This  holds  true  for  both  the  single  and  multiple 


Figure  3:  Amount  of  information  transferred  to 
the  operator  in  multiple  and  single  window 
conditions,  as  a  function  of  the  transition 
frequency.  Low  transition  frequencies  on  the  left 
side  correspond  with  longer  lasting  changes  in  the 
tactical  situation,  high  transition  frequencies  on 
the  right  side  with  short-lived  changes. 


window  set-up’s.  However,  an  analysis  of  variance 
(ANOVA)  on  the  transient  part  of  the  information 
transfer  (transition  frequency  about  0. 1  Hz)  showed 
a  significantly  higher  information  transfer  for  the 
multiple  windows  condition,  compared  to  the  single 
window  condition.  Furthermore,  as  a  measure  of 
effort,  all  switching  actions  related  to  changes  in 
display  settings  were  recorded.  The  results  are 
represented  in  figure  4.  An  ANOVA  showed  this 
number  to  be  significantly  lower  for  the  multiple- 
windows  condition  (150  for  the  MW  condition,  263 
for  the  SW  condition).  In  this  experiment  no 
specific  effects  could  be  found  related  to  warfare 
type  (AAW  en  ASW). 

Resuming,  results  of  the  experiment  show  that  the 
use  of  multiple  windows  has  a  positive  effect  on  the 
speed  of  detection  for  high-priority  targets.  Given 
the  frequency  with  which  changes  took  place  in  the 
scenarios,  subjects  had  less  time  to  pay  attention  to 
lower  priority  changes.  For  these  changes  it  took 
the  subjects  much  more  time  to  detect  them,  if 
detected  at  all,  and  as  a  consequence  differences 
between  the  single  and  multiple  windows 
conditions  became  less  pronounced. 

As  an  indication  of  required  effort  at  the  interface 
level  to  achieve  a  certain  detection  performance,  a 
substantial  reduction  of  user  actions  is  obtained  for 
the  multiple  windows  set-up.  Evidently,  for  the 
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Figure  4:  Number  of  operator  actions  related  to 
changes  in  range  selection  and  other  display 
settings  for  two  different  scenario  types  (AAW 
vs.  ASW),  tested  under  single-  and  multiple- 
window  conditions. 

multiple-windows  set-up  subjects  were  better 
supported  in  tailoring  the  information  presentation 
to  the  current  tactical  situation,  resulting  in  less 
'switching'  effort  required  for  this  tuning  process 
between  interface  and  environment. 


MULTIPLE  VIEWS 

As  described  in  the  previous  section,  information 
on  the  environment  in  current  command 
information  systems  is  presented  in  a  two- 
dimensional  ‘bird’s-eye  view’.  Regarded  as  much 
more  effective  than  the  older  generation  displays, 
new  advances  in  graphics  capabilities  and  display 
technology,  however,  may  offer  options  for  further 
improvement.  In  search  for  further  integration,  one 
of  these  options  is  certainly  the  application  of  3D 
perspective  or  stereoscopic  displays. 

The  strongest  motive  to  go  three-dimensional  is  the 
inability  to  combine  the  two  dimensional  ‘bird’s-eye 
view’  with  a  graphical  presentation  of  altitude  and 
depth  information.  As  a  consequence,  in  all  current 
systems  altitude  and  depth  information  is  presented 
as  numerical  read-outs.  According  to  Dennehy  et  al. 
[3]  representations  lacking  integrated  altitude  and 
attitude  information  complicate  situation  assessment 
in  two  ways.  Data  that  are  difficult  to  acquire,  are 
more  difficult  to  use  in  making  a  decision.  Also, 
without  immediately  evident  altitude  information,  a 
decision  maker  may  substitute  arbitrary  or  situation- 
biased  altitudes,  that  may  be  difficult  to  supplant 
even  when  the  actual  data  are  presented. 

With  more  realistic  images  of  the  environment  and 
tracks  in  a  3D  perspective,  they  argue,  the  interface 
becomes  more  natural  and  less  effort  is  required  to 
comprehend  the  current  situation.  It  eliminates  the 


Figure  5:  Example  of  simplified  tactical  picture  presented  on  a  3D  perspective  display,  with  air 
tracks  projected  on  the  surface  plane  to  amplify  precise  perception  of  position  and  altitude. 
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burden  of  integrating  and  interpreting  of  multiple 
representations,  abstract  symbols,  and  textual  read¬ 
outs.  Some  earlier  experimental  results  with 
perspective  displays  confirm  these  expectations.  In  a 
direct  comparison  between  a  conventional  and  a 
perspective  display  Bemis  et  al.  [4]  evaluated 
operator  performance  for  two  different  types  of 
tasks:  detect  threats  and  select  the  closest  interceptor 
for  each  detected  threat.  The  experiment  revealed  a 
significant  reduction  in  errors  of  detection  and 
interception  with  the  use  of  a  perspective  display. 
Also  response  time  for  selecting  interceptors  was 
greatly  reduced.  Indicating  the  potential  to  improve 
performance,  3D  perspective  or  stereoscopic 
displays,  however,  can  also  have  their  drawbacks. 
Inherent  to  the  perspective  view,  objects  are 
presented  larger  or  smaller  as  a  function  of  the 
operators  viewing  distance,  location  and  angle. 
Objects  close  to  the  operator  will  be  shown  with 
much  more  resolution  than  objects  at  larger  viewing 
distances.  In  many  cases  these  differences  will  not 
necessarily  reflect  differences  in  tactical  relevance 
and  meaning. 

One  can  also  question  how  accurate  altitude  or  depth 


Figure  6:  3D  perspective  split-up  in  2D  view 
from  above  and  side-view,  as  an  example  for 
what  is  defined  as  a  multiple  view  display. 


information  can  be  perceived.  To  prevent  errors,  and 
to  facilitate  more  precise  altitude  perception,  extra 
visual  cues  have  to  be  added  like  projections  on  the 
earth’s  surface  and  ruled  vertical  lines  connecting  the 
object  symbol  and  the  surface  (see  figure  5). 
Furthermore,  to  resolve  ambiguities  and  to  reduce 
clutter,  operators  should  have  hill  control  of  viewing 
distance,  angle  and  position  [5].  Although  this  will 
give  the  operator  the  flexibility  to  visualise  tactical 
data  more  freely  it  is  not  clear  what  effect  frequent 
changes  in  view  will  have  on  orientation  and 
situational  awareness. 

To  study  the  last  question  in  particular,  an 
experiment  was  conducted  to  test  how  solid  or 
reliable  the  internal  representation  of  a  tactical 
situation  is  when  the  situation  has  been  extensively 
explored  from  different  viewing  locations  [6],  Air 
and  surface  objects  in  the  explored  situation  were 
moved,  removed  or  interchanged  position.  Some 
changes  were  easy  to  recognise,  others  more  subtle. 
Shots  of  the  changed  situations  were  put  together 
with  shots  of  the  situation  as  explored,  and  subjects 
were  asked  to  judge  every  shot  on  whether  the 
situation  was  changed  or  not.  Performance  for  a  3D 
stereoscopic  display  was  compared  with  a  2D 
multiple-view  display.  This  2D  multiple  view  was  a 
composite  display,  with  a  side-view  added  to  the 
conventional  bird’s-eye  view  (see  figure  6).  It  lacks 
the  integration  of  the  3D  display,  but  has  the 
advantage  of  a  graphical  representation  of  altitude  or 
depth,  without  the  drawback  of  perspective 
distortions. 

Results  of  the  experiment  show  that  performance 
was  not  as  naturally  in  favour  of  an  integrated  3D 
display.  Subjects  in  the  2D  multiple-view  condition 
had  a  significant  higher  score  on  correct  identified 
shots,  being  the  tactical  situation  as  explored  or  a 
situation  that  had  changed  (see  figure  7). 

Analysis  of  data  from  a  second  experiment,  to  be 
reported  this  year,  show,  in  addition  to  this  result, 
that  differences  in  sensitivity  to  changes  especially 
hold  true  for  the  small,  more  subtle  changes. 
Regarding  response  time,  the  time  needed  to  identify 
a  tactical  situation  as  unchanged  (same)  or  changed 
(different),  responses  for  the  2D  multiple-view 
condition  had  the  tendency  to  be  a  little  bit  slower, 
but  the  difference  was  too  small  to  be  statistically 
significant. 

Although  more  experiments  are  needed  with  more 
tasks  to  be  tested,  the  results  put  some  doubts  on 
explanations  that  performance  improvements  as 
found  in  earlier  experiments  largely  depend  on 
integration  in  a  3D  perspective  display.  An  even 
better  performance  was  found  with  related,  but  non- 
integrated  multiple  views  of  the  tactical  situation. 
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Figure  7:  Percentage  of  correct  same-different 
answers  to  questions  testing  the  operator’s 
sensitivity  to  changes  in  the  tactical  situation. 
Scores  are  presented  as  deviations  from  the  50  % 
baseline,  associated  with  completely  random, 
non-sensitive  operator  behaviour. 

The  change  from  numerical  read-outs  to  the 
graphical  representation  of  altitude  or  depth  seems  to 
be  the  most  important  one  to  realise  a  more  effective 
information  transfer  and  an  improved  situational 
awareness.  With  integration  no  further 
improvements  were  obtained.  Disadvantages  of 
perspective  views  even  seem  to  have  a  suppressing 
effect  on  performance  accuracy. 

MULTIPLE  LAYERS 

The  most  obvious  application  of  3D  stereoscopic 
displays  is  to  get  an  almost  ‘natural’  representation 
of  objects  in  the  battlespace,  as  described  in  the 
previous  paragraph.  Stereoscopic  vision,  however, 
can  also  be  applied  for  the  visual  separation  of 
different  information  layers  with  each  layer 
containing  two-dimensional  representations. 

With  the  transition  from  vector  displays  to  raster- 
scan  displays  and  flat-panel  displays  in  the  near 
future,  together  with  fast  powerful  graphics 
processing,  the  potential  for  all  kinds  of  graphical 
representations  of  information  has  strongly 
expanded.  As  a  result,  many  categories  of 
information  can  be  brought  together  in  one 


integrated  graphical  representation  of  the 
environment  and  the  tactical  situation. 

To  prevent  problems  like  display  cluttering  or 
visual  interference  and  to  optimise  the  tactical 
picture  for  the  task  or  the  situation  at  hand,  new 
generation  systems  offer  extensive  filter  and 
display  options.  In  practice  however,  operators 
show  reluctance  to  use  the  flexibility  offered  where 
too  many  options  have  a  negative  impact  on 
overview  and  accessability,  and  extensive 
interaction  with  the  system  is  required,  especially 
under  time  pressure. 

Another  way  to  cope  with  this  problem  is  to  search 
for  well-balanced  visual  representations  in  which 
the  perception  of  objects  like  air  or  surface  tracks 
does  not  interfere  with  the  perception  of  supporting 
graphical  information  in  the  background.  Some  new 
opportunities  to  accomplish  a  better  visual 
seperation  between  foreground  and  background 
information  became  available  with  the  introduction 
of  full-colour  displays. 

An  example  of  the  difference  between  the  classical 
monochrome  tactical  display  and  a  full-colour 
tactical  display  was  already  given  in  the 
introduction  (figure  1).  This  example  of  the  full- 
colour  tactical  display  reflects  some  of  the 
information-presentation  principles  as  applied  to 
the  tactical  displays  on  board  the  M-class  frigates 
of  the  RNethN.  Through  the  combination  of 
information  presentation  in  both  positive  and 
negative  contrast,  a  visual  seperation  could  be 
realised  between  three  different  information  layers: 

•  area  filled  geographic  information  in  different 
shades  of  gray, 

•  primary  track  information  in  bright  colours  on 
the  foreground  in  negative  contrast  with  the 
display  background, 

•  and  supporting,  secundary  information  in  dark 
colours  on  the  foreground  in  positive  contrast 
with  the  display  background. 

Although  regarded  as  a  significant  improvement, 
the  possibilities  remain  limited  and  do  not  allow  to 
expand  to  a  more  full-scale  multiple-layer  model 
for  the  organisation  and  integrated  presentation  of 
information  as  presented  in  figure  8.  Experiments 
with  stereoscopic  displays  reported  in  the  previous 
paragraph,  however,  triggered  the  idea  to  present 
the  information  layers  at  separate  viewing 
distances.  It  will  allow  the  operator  to  visually 
focus  on  one  layer  in  the  context  of  information  in 
other  layers. 
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medium  for  the  selection  and  activation  of 
functions.  The  envisioned  outcome  of  this  study  is 
an  object-oriented  interface  design,  enabling  the 
user  by  direct  manipulation  of  ‘tactical  objects’  on 
the  tactical  display  to  modify  and  reconfigure  the 
tactical  workstation  for  an  effective  supervision  of, 
and  rapid  response  to  the  tactical  situation  at  hand. 


DISCUSSION  AND  CONCLUSIONS 


Figure  8:  Multiple  layer  concept  for  the 
organisation  and  integrated  presentation  of 
information. 

What  the  impact  will  be  on  performance  in  terms  of 
situational  awareness,  assessment  and  decision 
making  is  still  unknown.  Many  research  questions 
have  to  be  answered.  As  part  of  a  research  program 
on  emerging  interface  technologies  and  their 
application  in  command  and  control,  preparations 
have  been  started  to  build  a  concept  demonstrator, 
to  have  the  tools  to  test  and  evaluate  the  potential  of 
such  a  tactical  display. 


TACTICAL  OBJECTS 

Besides  the  increasing  amount  of  available 
information,  present-day  combat  management 
systems  offer  a  large  variety  of  functions  and 
services,  possibly  set  up  in  a  flexible  way  for  the 
tuning  of  this  functionality  to  the  different  operator 
roles.  However,  a  possible  negative  side-effect  in 
this  approach  may  be  the  rather  diverse  ways  in 
which  the  required  information  for  important  tasks 
such  as  threat  evaluation,  assignment  of  sensor/ 
weapon  systems,  and  weapon  deployment,  is 
divided  over  different  system  services.  Thus,  it 
requires  a  certain  amount  of  mental  effort  to 
maintain  overview  of  the  'flow  of  information', 
system  settings,  filters  etc.  This  problem  may  be 
enlarged  by  the  variety  of  functions  and  services 
which  are  available  through  various  (a.o.  soft-key 
type)  input  devices,  with  a  lack  of  context 
sensitivity. 

Within  this  scope,  possibilities  for  innovation 
currently  are  under  study,  exploring  the  way  in 
which  functions  may  be  represented  and  organised 
in  future  tactical  workstations.  Based  on  the  'select 
then  operate'  principle,  system  effectiveness 
possibly  may  be  increased  by  considering  the 
tactical  display  not  only  to  be  an  output  medium  for 
the  presentation  of  tactical  data  but  also  an  input 


Within  the  next  generation  combat  information 
centre,  command  teams  will  have  to  interact  with 
complex  information  processing  systems.  Data 
from  many  sources  in  large  databases  have  to  be 
selected,  combined  and  manipulated.  From  a 
warfare  officer's  point  of  view,  usability  and 
accessibility  are  keywords  of  primary  importance. 
Easy  access  strongly  depends  on  how  the 
environment,  situation,  plans  and  system  states  are 
visualised,  and  the  way  these  views  can  be 
manipulated.  New  concepts  and  technologies  for 
graphic  information  presentation  and  object 
manipulation,  both  two-  and  three-dimensional, 
already  are  or  will  become  available 
In  this  paper  some  new  concepts  have  been 
introduced.  The  common  denominator  seems  to  be 
the  word  multiple :  multiple  windows,  multiple 
views  and  multiple  layers.  They  all  reflect  the 
problem  of  integrating  growing  information  flows 
in  one  view  or  window  known  as  the  tactical 
picture. 

Results  of  experiments  indicate  how  the 
organisation  of  information  in  separate  windows, 
views  and  layers  can  improve  information  transfer 
and  situational  awareness. 

The  way  information  is  organised,  however,  also 
depends  on  the  task  and  the  tactical  situation  at 
hand.  The  next  challenge  is  to  develop  a  tool  for 
easy  re-organisation  of  information  display  when 
operators  have  to  switch  between  tasks  or  when 
changes  in  the  tactical  situation  take  place. 
Development  and  testing  of  the  tactical  object  as  an 
organising  principle  is  regarded  as  the  first  step  to 
meet  this  challenge. 
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Summary:  Battlespace  Management  systems  are  often 
developed  by  decomposing  the  problem  into  separate 
functions.  For  example,  the  battle  scene  is  decomposed 
into  intelligence  reports,  sensor  displays  for  each 
sensor,  contact  tracks  for  each  sensor  contact, 
environmental  (weather,  oceanography)  conditions  and 
predictions,  sensor  effectiveness  predictions, 
geophysical  /  physical  oceanographic  pictures,  etc. 
Once  the  problem  has  been  decomposed  and  analyzed, 
the  decision  maker  must  put  it  back  together  in  a 
mental  information  fusion  process,  integrating 
information.  The  tools  to  help  the  expert  decision 
maker  re-fuse  the  problem  are  far  fewer  and  more 
difficult  to  develop  than  the  tools  to  decompose.  The 
research  reported  here  takes  an  alternative  approach  by 
providing  information  displays  that  cluster  and 
integrate  information  according  to  the  expert  decision 
maker’s  knowledge  schema  and  procedural  structure.  A 
complex,  time-dependant  (but  non-military)  test 
domain  with  multiple,  conflicting  goals  was  selected. 
Functional  partitioning  required  greater  effort  while 
procedurally  based  information-clustering  resulted  in 
more  efficient  (timely  and  accurate)  decision  making. 

INTRODUCTION 

The  Problem:  Battlespace  Management  systems  are 
often  developed  by  partitioning  the  problem  into  many 
functions  or  tasks.  For  example,  one  tool  displays  all 
the  broadband  noise,  sensed  in  all  directions.  Another 
provides  a  similar  display  of  narrowband  noise 
received  by  each  sensor.  Another  tool  makes 
environmental  (atmospheric  or  oceanographic) 
predictions  for  specific  variables  (wind  /  current  speed, 
temperature  profile,  etc.).  Yet  another  tool  gives  the 
current  conditions.  Several  algorithms  work  to  solve 
the  target-motion-analysis  problem.  Thus  each  unit  in 
the  battlespace  is  decomposed  into  many  separate 
signals  and  kinematic  components,  all  independent 
from  one  another  and  from  their  underlying  physical 
location  and  the  constraints  that  it  imposes. 
Decomposition  facilitates  efficient  engineering  of  the 
algorithms  and  programs.  Each  function  has  its  own  set 
of  developers,  and  therefore,  its  own  set  of  tools.  The 
consequence  of  this  approach  is  that  the  problem  is 


split  into  many  different  unassociated  bits  of 
information.  However,  the  partitioning  scheme  is  not 
necessarily  congruent  with  the  way  that  the  decision 
maker  solves  the  problems. 

The  task  of  the  decision  maker  is  to  evaluate  the 
available  information,  predict  the  effects  of  various 
action  options,  and  communicate  the  decision. 
Evaluation  includes  integrating  data  from  the  different 
sources  described  above,  comparing  conditions  to 
assumptions,  assessing  accuracy,  etc.  The  available 
information  is  composed  partially  of  the  output  of 
various  tools  and  partially  of  unanalyzed  or  ‘Taw”  data. 
It  includes  history,  current  state,  and  predictions.  It 
may  contain  considerable  uncertainty  and/or  may 
change  over  time.  Required  decisions  may  include  both 
what  to  do  and  when  to  act.  Information  management 
tools  and  decision  aids  are  developed  for  these  tasks 
because  they  are  so  complex  and  because  the 
supporting  information  is  so  complex  and  uncertain. 
However,  once  the  problem  has  been  decomposed  and 
analyzed,  the  decision  maker  must  put  it  back  together 
in  a  mental  information-fusion  process,  integrating 
information  from  these  many  tools  and  phases.  The 
tools  to  help  the  expert  decision  maker  to  put  the 
problem  back  together  are  far  fewer  and  more  difficult 
to  develop  than  the  tools  to  take  it  apart. 

A  Proposed  Solution:  An  alternative  approach  to 
decision  aid  development  is  to  start  with  an 
understanding  of  the  knowledge  and  procedures  of  the 
expert  decision  maker  and  then  design  tools  to  support 
these.  With  this  knowledge,  we  could  design 
information  management  decision  aids  in  the  way  that 
new  walkways  are  sometimes  planned.  That  is,  where 
natural  paths  occur  because  of  repeated  use  by 
pedestrians,  constructed  (concrete,  macadam,  gravel, 
etc.)  pathways  are  built.  In  the  same  way,  battlespace 
management  tools  and  decision  aids  should  provide 
support  for  knowledge  in  the  head  —  the  procedural 
paths  we  create  through  the  task  and  the  information. 

Experts  make  use  of  the  procedural  components  of 
their  knowledge,  as  well  as  the  declarative  content: 
That  is,  they  know  how-to  as  well  as  what,  why,  when, 
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and  where.  Tools  used  to  do  a  task  and  procedural 
knowledge  of  that  task  are  not  independent  entities. 
The  tool  can  facilitate  the  task  procedures  or,  in  the 
case  of  clumsy  automation  (Wiener,  1989),  dictate 
conflicting  procedures.  The  organization  of 
information  can  provide  the  cognitive  equivalent  of 
affordances  (Norman,  1988)  or  “handles”  that  facilitate 
performance  or  obstacles  that  hinder  it.  For  example, 
calculators  can  use  either  arithmetic  notation  or 
“reverse  Polish”  notation.  Arithmetic  notation  allows 
the  user  to  enter  numbers  and  operators  as  they  would 
on  paper  (3  +  2  =)  and  supports  the  average  user’s 
procedural  knowledge.  Reverse  notation  requires  the 
user  to  enter  numbers  first  and  then  operators  (32  +  =). 
Although  this  notation  groups  like  information 
(numerals,  operators)  together,  it  requires  the  average 
user  to  reorder  the  information  from  normal  arithmetic 
procedures. 

There  is  a  well  documented  interaction  between 
knowledge  in  the  head  and  information  in  the  world. 
Kleinmuntz  and  Schkade  (1993)  reviewed  several 
studies  that  show  how  problem  representation  affects 
the  speed  and  accuracy  of  identifying  and  assessing  the 
situation,  and  consequently,  the  quality  of  the  decisions 
made  with  that  information.  For  example,  Johnson, 
Payne  and  Bettman  (1988)  found  that  display  format 
effects  the  likelihood  of  preference  reversals  (a  well- 
documented  decision  error)  in  choice  decision  making. 
Decision  makers  in  these  studies  shifted  information 
gathering  strategies  as  a  function  of  display  format. 
Brown  and  Klayman  (1989)  and  Smith  (1989)  found 
that  representation  affects  subjects'  ability  to  identify 
key  problem  elements  in  naturalistic  decision 
situations.  Larkin  (1989)  has  called  this  effect  display- 
based  problem  solving  because  the  availability  and 
form  of  the  information  displayed  can  affect  problem 
solving.  For  example,  Russo  (1977)  found  that  a  table 
of  unit  prices  for  an  entire  category  of  food  facilitated 
price  comparison  and  decision  making  as  compared  to 
unit  prices  displayed  with  each  item,  although  unit 
prices  are  calculated  by  item,  not  category.  One  reason 
for  this  improvement  may  be  the  reduction  in  working 
memory  load  when  appropriate  information  is 
clustered.  Thus,  the  tools  used  to  do  a  task  and 
procedural  knowledge  of  how  to  do  that  task  are  not 
independent  entities.  The  tool  can  facilitate  the  task 
procedures  or,  in  the  case  of  clumsy  automation 
(Wiener,  1989),  dictate  conflicting  procedures. 

To  solve  the  problem  posed  above  I  propose  providing 
information  displays  that  cluster  and  integrate 
information  according  to  the  expert  decision  maker's 
knowledge  schema  and  procedural  structure  rather  than 
according  to  a  functional  one.  I  hypothesize  that  such  a 
system  would  lead  to  more  efficient  decision 
performance.  What  I  mean  by  efficient  is  equal  or 
better  performance  in  a  shorter  time,  with  less  effort. 


In  the  remainder  of  this  chapter  I  will  first  report  on  an 
experiment  that  demonstrates  the  performance 

advantages  for  this  idea  and  then  I  will  discuss  several 
military  applications  for  the  findings. 

TESTING  THE  HYPOTHESIS 

To  test  the  hypothesis  a  complex,  time-dependant  (but 
non-military)  test  domain  with  multiple,  conflicting 
goals  was  selected.  The  decision  task  was  designed  to 
have  a  one-to-one  correspondence  with  key  elements 
of  the  submarine  problem.  (One  advantage  of  the  non¬ 
military  task  was  that  many  more  experts  were 
available.  Additional  evaluation  verified  the  task 
validity.)  Three  information  format  schemes, 

alphabetical  listing  (format  A),  functional  partitioning 
(format  B),  and  procedurally  based  information¬ 

clustering  (format  C)  were  tested.  Version  C  was 
designed  by  a  bootstrapping  procedure  based  on  two 
individuals  pilot  testing  versions  A  and  B. 

Three  classes  of  dependent  measures  were  used.  The 
first  was  total  time-on-task.  The  second  reflected 
outcome  performance,  and  the  third  measured 
processing  activity.  Results  showed  that  version  C  lead 
to  the  most  efficient  performance.  There  was  an 
interaction  between  performance  measures  and 
measures  of  processing  time  and  processing  effort. 
Functional  partitioning  required  greater  effort  for 
limited  performance  improvement  over  the 
alphabetical  format.  Thus,  the  right  organization 
scheme  can  provide  the  support  for  improved  cognitive 
performance. 

There  are  many  possible  partitioning  schemes  for 
categorizing  and  organizing  information.  Like  the 
contents  of  a  computer  directory,  tools  on  a 
workbench,  or  merchandise  in  a  store,  information  in  a 
decision  support  system  can  be  organized  by  many 
attributes,  including  size,  purpose,  time,  or  order  of 
use.  The  different  organizational  structures  facilitate 
achieving  different  goals.  Random  placement  speeds 
cleaning-up  after  a  project,  but  organization  by  purpose 
speeds  retrieval  of  tools  from  a  workbench. 

Phase  1:  Information  Organization:  The 

experimental  task  was  simulated,  on-line,  college 
course  scheduling.  This  task  has  many  elements  in 
common  with  the  target  task,  dynamic  decision  making 
under  uncertainty,  but  has  many  more  experienced 
individuals  to  serve  as  testers.  To  simulate  an  event- 
driven  environment,  classes  could  fill  while  the 
"student''  was  selecting  a  schedule.  When  a  planned 
course  was  filled,  the  subject  had  to  reassess  the 
situation  and  find  a  new  course  that  fulfilled  the  other 
requirements.  Elements  of  data  history  were  important 
because  previous  semester's  records,  program 
requirements,  and  course  prerequisites  had  to  be 
reconciled.  A  set  of  sometimes  conflicting  goals  further 
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constrained  choices  (see  Table  1).  Lastly,  of  course, 
classes  could  not  conflict  with  one  another.  To 
simulate  the  multiple  sources  of  information,  each 
function  (e.g.,  instructor  rating  lists,  course  schedules, 
requirements  lists,  student  history,  course  locations  and 
distance  maps,  etc.)  had  its  own  information 
presentation.  This  task  is  not  a  simple  scheduling 
problem  because  there  is  no  single  “optimal”  or 
algorithmic  schedule  that  solves  all  constraints  and 
meets  all  the  goals.  It  requires  goal-driven  decision 
making  to  achieve  acceptable  performance. 

A  scoring  scheme  was  developed  that  operationalized 
each  of  these  elements  as  values  associated  with 
accomplishing  the  goals  and  values  for  each  of  the 
choices  (i.e.,  courses  and  instructors).  It  was  predicted 
that  scores  would  reflect  the  level  of  organization  in 
the  display  schema. 

_ Table  1 :  Goals,  listed  in  priority  order 

■  Register  for  15-17  hours  (5  courses). 

■  Try  to  fulfill  requirements  and  prerequisites  for 
both  general  education  and  your  major.  (Note 
that  you  may  remain  an  industrial  engineering 
major  or  select  engineering  psychology.) 

■  Try  to  schedule  so  that  you  have  one  full  day 
or  two  half-days  off.  (One  full  day  is 
preferable.  Assume  you  have  a  job  and  will 
otherwise  have  to  work  on  the  weekend.) 

■  Try  to  get  the  best  instructors  possible.  A  list  is 
provided  of  instructor  ratings  from  a  student 
survey. 

■  Try  to  avoid  8  am  classes. 


Information  in  each  of  the  sources  was  structured 
differently. 

Although  the  majority  of  these  materials  were 
organized  by  academic  department,  other  schemes 
were  also  used,  including  listings  by  time,  by  course 
number,  and  alphabetically,  by  name.  Version  A  was 
intended  to  serve  as  a  baseline  of  minimal  performance 
and  maximal  task  time.  Armchair  analysis  suggested 
that  organization  alphabetical  listings  by  course  title 
and  listing  by  course  number  was  unlikely  to  match 
anyone’s  procedural  or  declarative  knowledge  schema 
for  this  kind  of  information.  Thus,  it  was  used  for 
version  A. 

For  Version  B,  recall  and  interview  with  individuals 
who  had  attended  college  prior  to  the  computerization 
of  registration  led  to  organization  by  information  type 
(student  record,  departmental  course  listing, 
requirements,  etc.),  much  as  it  had  been  in  my  student 
days.  To  prevent  the  participants  in  Conditions  A  and 
B  from  reorganizing  the  information  by  opening 
multiple  windows,  the  metaphor  of  an  electronic  book 
was  used,  with  “pages”  for  each  kind  of  information 
above.  The  book  could  only  be  open  to  one  page  at  a 
time. 


Apparatus  Design  Methods:  Establishing  the  correct 
information  structure  can  be  an  iterative,  almost 
circular  process  as  procedures  can  change  when  tools 
change.  The  question  herein  is  not  how  to  design 
displays,  but  what  are  the  effects  of  information 
organization  on  performance.  Three  prototype 
information  presentation  schemes  were  developed. 
Versions  A  and  B  were  modeled  on  the  registration 
materials  used  by  many  pre-internet  generations  of 
students.  These  included  student  record;  a  catalogue 
listing  general  degree  requirements  and  specific 
requirements  for  each  major,  course  descriptions; 
prerequisites  for  each  course;  a  schedule  of  courses  for 
the  upcoming  semester;  a  student  schedule  sheet  for 
recording  selections;  and  an  informal  rating  of  faculty 
published  in  the  student  newspaper.  Students  had  to 
determine  the  availability  of  seats  for  each  course,  by 
going  to  each  department.  For  this  experiment,  this  last 
step  was  consolidated  into  a  single  list.  Selecting 
courses  required  multi-way  comparisons  among  the 
sources  of  information,  and  across  many  pages  in  each. 
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Figure  1 :  Steps  used  by  a  typical  student  to  complete  a 
course  schedule. 


Version  C  was  modeled  on  modem  computer 
registration.  This  was  designed,  not  by  attempting  to 
recreate  a  historic  artifact,  but  by  a  bootstrapping 
procedure  based  on  two  individuals  pilot  testing 
versions  A  and  B.  Figure  1  shows  all  of  the  steps  used 
by  these  individuals1.  The  many  steps  used  different 
subsets  of  the  available  information  (see  Figure  1). 
These  subsets  were  used  sequentially,  as  a  single 
cluster,  although  the  order  was  not  invariant.  For 
example,  two  clusters  were  used  to  make  initial  course 
selections.  These  were  (1)  requirements  and  student 
record  of  courses  already  taken,  and  (2)  course 
prerequisites,  student  record,  and  times  of  potential 
courses. 

The  same  information  could  be  used  by  several  steps. 
To  accommodate  all  possible  steps  in  this  procedural 
approach  to  information  organization,  the  clusters  of 
information  most  often  used  by  a  single  step  were 
displayed  in  the  same  window  or  in  other  windows  that 


could  be  open  simultaneously.  Thus,  only  eye 
movements  were  necessary  to  obtain  all  the 
information  used  by  any  step. 

PHASE  2  BEHAVIORAL  EXPERIMENT 
Participants:  The  two  a  priori  requirements  for 
participants  were  that  they  be  recent  college  students 
with  a  minimum  of  eight  semesters  and  that  they  had 
registered  for  college  classes  within  the  past  five  years. 
They  could  be  considered  experienced  at  the  putative 
task.  There  were  10  women  and  26  men  evenly 
distributed  across  the  three  conditions.  They  had  last 
registered  for  college  courses  an  average  of  2.86  years 
prior  to  the  experiment.  Educational  level  of  the 
sample  ranged  from  bachelors  to  doctorate  degree.  The 
experiment  used  a  between  subjects  design  with  twelve 
participants  per  group. 

Apparatus  and  materials:  The  task  was  a  second  year 
college  registration.  Participants  were  given  goals, 
student  records  of  course  and  grade  history, 
prerequisites,  a  campus  map  with  walking  times,  and 
instructor  ratings.  The  task  was  timed  and  courses 
closed,  dependent  on  the  elapsed  time.  All 
experimental  material  was  developed  in  Supercard  and 
presented  on  a  Macintosh  computer  with  a  19  inch 
color  display.  Conditions  A  and  B  used  a  booklet-like 
format  with  only  a  single  page  visible  at  a  time.  The 
page  numbers  in  the  Table  of  Contents  were  hypertext 
links  to  the  listed  information.  Categories  are  listed  in 
Table  2. 

Table  2:  Categories  of  information  in  Table  of 
Contents 

■  Goals 

■  Student  record 

■  General  requirements 

■  Departmental  requirements  for  Major 

■  Course  Prerequisites 

■  Courses  Schedule 

■  Campus  map 

■  Table  of  Instructor  ratings 

■  Table  of  class  size  and  seats  remaining 

In  condition  A,  all  information  in  the  course  schedule 
and  prerequisites  sections  was  listed  in  alphabetical 
order.  Thus,  Introduction  to  Psychology  followed 
Introduction  to  Physics.  As  courses  were  not  listed  by 
course  title  (only  course  number)  in  the  requirements 
and  student  record  sections,  this  format  required  a 
search  information  retrieval  strategy.  In  condition  B 
course  schedule  and  prerequisites  were  listed  by 
department,  and  sequentially  by  number  within 
department. 


1  Not  all  steps  were  used  on  every  trial. 
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Condition  C  used  the  procedural  format  developed  in 
Phase  1 .  It  used  a  computer  registration  analogy  with 
access  to  information  via  a  menu.  Multiple  movable 
windows  could  be  open  simultaneously.  These  were 
scrollable  and  resizable  when  required  (e.g.,  course 
schedule,  registration  card,  and  any  other  card  with 
more  than  about  10  lines  of  information).  As  the  screen 
and  text  were  of  the  same  size,  approximately  the  same 
quantity  of  information  was  visible  in  all  conditions. 

In  all  three  versions  of  the  task,  the  program  recorded 
windows  opening,  buttons  being  pushed,  typed  text, 
and  the  time  (in  ticks)  associated  with  each  interaction. 

Procedure:  The  experiment  took  place  in  a  small, 
sound-damped,  experimental  room.  All  instructions 
were  presented  on  the  computer  screen.  Participants 
were  given  introductory  instruction  on  manipulation  of 
the  objects  used  in  the  program  and  on  the  task.  There 
was  a  practice  task  for  each  condition  that  duplicated 
all  of  the  screens,  interactions,  and  information  types. 
During  the  practice  and  before  the  actual  experimental 
trial,  participants  were  invited  to  ask  questions, 
however,  questions  about  strategy  were  not  answered. 
No  questions  were  answered  after  the  experimental 
trial  began.  At  the  end  of  the  experimental  task, 
participants  were  asked  to  complete  a  computerized 
questionnaire  and  were  debriefed.  The  questionnaire 
was  designed  to  ascertain  recall  of  relevant 
information,  task  strategy,  computer  and  college 
registration  expertise,  and  any  comments  about  the 
experiment. 

STATISTICAL  RESULTS 

Three  classes  of  dependent  measures  were  used.  The 
first  was  total  time-on- task  (T).  The  second  reflected 
outcome  performance  (P),  and  the  third  measured 
processing  activity  (A). 

Performance  was  defined  as  the  summation  of  the 
following  four  performance  measures:  the  number  of 
credits  successfully  registered  (Pj),  the  sum  of  the 
requirements  scores  for  all  courses  registered  (P2),  the 
sum  of  the  scheduling  difficulty  scores  for  all  courses 
registered  (P3),  and  the  average  preference  score  for  the 
instructors  of  all  selected  courses  (P4).  Scheduling 
difficulty  was  computed  as  the  number  of  seats  in 
courses  that  would  satisfy  requirements  times  the 
number  of  credits,  weighted  by  the  scheduling 
priorities  given  in  the  goal  list.  Each  of  the  four  scores 
was  determined  a  priori  and  was  reflected  in  the  goal 
set  given  to  the  participants. 

The  processing  measures  captured  various  aspects  of 
the  effort  participants  put  into  the  task.  These  included, 
number  of  registration  attempts  (At),  number  of  class 
close-outs  (A2),  number  of  windows  used  (A3),  and 
number  of  times  the  subject  iterated  back  and  forth, 


between  any  two  windows  (A4).  One  full  cycle  from 
window  a  to  b  to  a  to  b  was  counted  as  one  iteration. 

These  measures  were  combined  into  an  overall 
efficiency  measure.  Efficiency,  E,  was  defined  as  the 
ratio  of  mean  overall  performance,  to  mean 

amount  of  processing,  plus  total  time-on-task 

required  to  achieve  that  level  of  performance,  T: 

E  =  m^yCm^+T)  (1) 

The  groups  did  not  differ  on  any  of  the  measures  of 
computer  experience  or  college  registration  experience. 
Table  3  shows  means  and  standard  deviations  for  all 
behavioral  measures  (T,  P,  A,  &  E).  To  facilitate 
comparisons  among  measures,  all  scores  were 
transformed  into  standard  scores  with  a  mean  of  50  and 
a  standard  deviation  of  10. 

Time-on-task:  While  means  were  not  significantly 
different  among  groups,  variances  were  large,  time-on- 
task  did  contribute  to  individual  performance 
differences.  To  account  for  differences  in  time  taken  by 
individual  participants,  performance  and  processing 
measures  were  computed  per  unit  time.  While  not 
statistically  different  due  to  large  variances,  the  trend 
was  surprising.  Versions  A  and  C  appeared  equally  fast 
(Figure  2). 


Group 


Figure  2:  Mean  time-on- task  and  mean  performance. 

Performance  Measures:  Overall  performance, 

superimposed  over  time-on-task  shows  the  relationship 
between  the  two.  Those  using  version  A  found  it  so 
difficult  that  they  basically  gave  up  trying  perform 
well.  They  just  wanted  to  complete  the  task  quickly. 
People  using  version  B  found  that  they  could  complete 
the  task,  but  it  took  considerable  time  and  effort. 

Decomposing  the  performance  measures  provides  the 
supporting  detail  necessary  to  understand  this  effect. 
The  performance  measures  appeared  to  be  composed 
of  two  compound  measures  that  behaved  very 
differently  from  one  another  (see  Figure  3).  The  first, 
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P12,  was  composed  of  the  more  concrete  performance 
measures;  P,,  the  number  of  credits  successfully 
registered  and  P2,  the  sum  of  the  requirement 
satisfaction  scores  for  all  registered  courses.  The  tasks 
represented  by  these  measures  were  essential  for 
completion  of  the  course  schedule  and  did  not  reflect 
differences  in  performance.  They  replicated  minimal  or 
baseline  performance.  Although  there  appears  to  be  a 
slight  trend  toward  better  performance  for  groups  B 
and  C,  this  was  not  significant,  F<1.0. 


Figure  3:  Mean  performance  on  two  compound 
measures  for  each  display  format. 


Processing  Measures:  The  process  measures  also 
showed  two  patterns  of  behavior  (see  Figure  4).  For 

Table  3:  Means  and  standard  deviations  for  all 


measures  for  all  groups. 


Alphabetical 
M  SD 

Functional 

M  SD 

Procedural 

M  SD 

Time  on 

task 

49.24 

9.76 

52.54 

10.62 

48.21 

9.94 

Performance  Measures 

p, 

48.40 

11.77 

50.97 

6.36 

50.60 

11.62 

p2 

48.33 

12.20 

50.17 

8.87 

51.52 

9.23 

p3 

49.65 

10.07 

47.69 

11.35 

52.59 

8.73 

P4 

49.67 

11.46 

46.61 

8.60 

53.82 

9.29 

Process  Measures 

A, 

48.89 

5.43 

53.47 

15.04 

47.21 

5.76 

a2 

51.01 

12.33 

52.15 

10.93 

46.44 

3.95 

a3 

55.78 

8.98 

54.21 

7.60 

40.02 

3.92 

A4 

56.65 

9.65 

53.84 

6.37 

39.51 

0.42 

Efficiency  Ratio 


ease  of  analyses  ar$?  discussion,  Si'ese  sha?I^e  cafled 
A,  2  and  A3  4,  with  the  understanding  that  the  two 
components  of  each  compound  measure  displayed  the 
same  pattern  of  results.  The  first  compound  measure 
was  composed  of  measures  A1?  registration  attempts 
and  A2,  number  of  close-outs.  These  are  both 
indications  of  difficulties  with  the  task,  rather  than  the 
information  format.  There  were  no  significant 


differences  among  the  conditions  on  this  compound 
measure,  F(2,33)  =  1.71,  n.s. 


Figure  4:  Mean  processing  score  on  two  compound 
measures  for  each  display  format. 


The  second  compound  performance  measure,  P3?4, 
was  composed  of  more  difficult,  evaluative  and 
integrative  tasks;  P3,  the  sum  of  the  scheduling 
difficulty  scores  for  all  courses  registered,  and  P4,  the 
average  preference  score  for  the  instructors  of  all 
selected  courses  (see  again  Figure  3).  These  measures 
appeared  to  reflect  an  added  effort  to  perform  well, 
when  possible.  P3  4  showed  a  paradoxical  dip  in 
performance  with  the  functional  display  organization, 
B.  This  quadratic  trend  was  marginally  significant, 
F(l,33)=4.07,  p=  05. 

The  second  pair  of  processing  measures;  A3,  number  of 
windows  and  A4,  number  of  iterations,  are  related  to 
information  accessibility  and  congruence  with 
procedural  needs.  If  the  information  format  does  not 
match  the  sequences  used  by  procedural  knowledge, 
the  individual  must  collect  it  from  where  it  is 
(indicated  by  A3)  and  then  create  the  sequence  in 
working  memory  (encoding  and  sequencing  indicated 
by  A4).  There  were  significant  differences  among  the 
groups  on  this  pair  of  measures,  F(2,33)  =  43.04, 
p<.001.  This  was  a  very  robust  effect  with  q2  =  0.72 
and,  in  a  post  hoc  test  for  trend,  the  quadratic  trend  was 
significant,  F(l,33)  =  13.104,  p<.005. 

Efficiency:  The  processing,  performance  and  time-on- 
task  measures  combined  to  evaluate  the  effect  of 
information  organization  of  decision  making 
efficiency.  Participants  who  used  the  procedural 
information  format  were  significantly  more  efficient  in 
their  decision  making  than  were  those  using  either  of 
the  other  two  formats,  F(2,33)=7.29,  p<.005  (see 
Figure  5).  This  was  a  robust  effect,  with  q2  =  0.32.  In  a 
post  hoc  test  for  trend,  the  quadratic  trend  was 
significant,  F(l,33)  =  4.32,  p<.05. 
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Figure  5:  Efficiency  (performance  divided  by 
processing  effort  and  time)  for  each  of  the  three  display 
formats. 

PROCEDURAL  ANALYSIS 

The  first  task  of  the  procedural  analysis  was  to  evaluate 
the  relationship  between  overall  performance  and 
overall  procedural  processes.  This  relationship  can  best 
be  understood  by  examining  Figure  6.  As  can  be  seen, 
there  was  an  inverse  relationship  among  these 
measures.  Processing  variables  were  moderately 
predictive  of  total  performance  score,  R2=  0.42. 


Figure  6:  The  relationship  between  performance  and 
process  measures  for  each  format. 

Iterations:  The  iteration  measure  is  a  reflection  of  the 
efficiency  of  procedures  used  by  participants  under 
each  of  the  three  conditions.  In  conditions  A  and  B, 
participants  physically  iterated  between  pairs  of 
information  while  in  condition  C  they  typically 
positioned  information  windows  so  that  information 
used  in  the  current  step  could  be  accessed  with  eye 
movements  and  did  not  require  mouse  actions.  A 
typical  pattern  for  the  participants  in  condition  A  was 
to  iterate  between  the  schedule  page  and  virtually 
every  other  page.  However,  there  were  numerous 
iterations  among  other  pages.  For  participants  in 
condition  C,  virtually  all  of  the  iterations  were  between 
the  schedule  page  and  one  of  the  pages  listing  course 


schedules.  There  were  very  few  iterations  for 
participants  in  condition  C  and  these  were  different  for 
each  subject.  Figure  7  shows  typical  patterns  of 
iterations. 


Figure  7:  Typical  patterns  of  iterations  for  participants 
in  each  of  the  three  conditions.  Node  #1  is  Table  of 
Contents. 


Sequences:  Just  as  the  number  and  pattern  of  iterations 
differed  by  condition,  other  aspects  of  performance 
differed  by  condition.  Unfortunately,  neither  verbal 
protocols  nor  eye  gaze  data  were  collected  so  one  can 
only  infer  goals.  Self  reports  in  the  follow-up 
questionnaire  shed  no  light  on  the  question  because 
there  were  no  differences  among  groups  and  because 
participants  often  reported  placing  no  weight  on  a 
source  of  information  that  they  examined  frequently  or, 
conversely,  placing  heavy  weight  on  information  that 
they  never  accessed. 

Sequences  of  pages  (Conditions  A  and  B)  or  windows 
(Condition  C)  provide  insight  into  goals  and  validation 
of  the  procedure  described  in  Figure  1 .  Several  of  the 
sequences  were  found  in  all  conditions  including  those 
used  for  procedures  1  and  2.  Only  participants  in 
Condition  C  were  able  to  see  a  listing  of  courses  by 
time  slot.  The  majority  of  them  used  this  display,  but 
only  for  the  last  one  or  two  course  selections. 

The  sequence  of  the  requirements  listings  either 
preceded  or  followed  by  student  record  (procedural 
step  1,  Figure  1)  is  used  to  illustrate  differences  among 
the  groups.  Groups  A  and  B  examined  requirements  an 
average  of  5.85  (SD  =  1.29)  and  6.75  (SD  =  0.94) 
times,  respectively  while  group  C  reviewed  this 
important  information  an  average  of  10.75  (SD  =  1.01) 
times.  Moreover,  review  of  this  pair  of  windows  was 
not  evenly  distributed  across  the  duration  of  the  task.  It 
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appeared  more  frequently  prior  to  early  course 
selections.  Apparently  as  the  possibilities  narrowed, 
many  participants  chose  to  skip  step  1  in  the  procedure. 

DISCUSSION 

Providing  guidance  for  the  development  of  interfaces 
that  support  efficient  (proficient  and  timely)  decision 
making  was  a  major  motivator  for  this  study.  The  most 
obvious  conclusion  is  that  these  results  indicate  the 
importance  of  information  organization  that  is 
congruent  with  procedural  knowledge.  Moreover,  they 
show  the  impact  of  such  organizational  schemes  on 
efficient  performance.  A  more  detailed  examination  of 
the  data  indicates  that  differences  in  support  for 
procedural  knowledge  differentially  change  task 
procedures.  Evidence  for  these  differences  are 
provided  by  all  three  sets  of  results  reported  here, 
performance  score  components,  processing  activity 
components,  and  the  iterations  picture. 

Performance:  Differences  in  the  two  compound 
performance  scores,  Vl  2,  (the  more  concrete,  required 
performance  measures)  and  P34,  (the  optional 
evaluative  and  integrative  measures)  suggest  that 
motivation  plays  a  subtle  role  in  the  equation. 
Performance  on  the  P,  2  measure  was  flat,  but  with 
large  individual  differences.  A  ceiling  effect  may  have 
contributed  to  the  lack  of  systematic  differences. 

P3  4  performance  was  clearly  affected  by  some  aspect 
of  the  tool.  As  these  measures  related  to  qualitative 
performance,  they  might  reflect  the  ease  of  use  for  the 
different  organizational  schemes.  Clearly,  Version  C 
provided  the  cognitive  affordances  for  better 
performance.  The  processing  scores  (see  below) 
contribute  to  this  conclusion. 

Surprisingly,  performance  was  not  poorer  on  any 
measure  for  Condition  A.  Although  that  version  of  the 
task  was  intended  to  provide  the  least  support  for 
procedural  knowledge,  it  did  not  differ  from  the 
traditional,  department  (like-with-like)  organizational 
scheme.  Might  it  be  true  that  any  information 
organization  that  is  not  congruent  with  procedural 
knowledge  restricts  performance? 

Processing  Activity:  As  with  the  performance 
compound  measures,  differences  in  the  two  compound 
processing  measures  showed  different  patterns. 
Compound  measure  A,  2  reflects  the  task  difficulty  and, 
not  surprisingly,  did  not  differ  among  the  three 
conditions.  This  lack  of  difference  verifies  that  the  task 
could  be  accomplished  with  any  of  the  three  versions. 

Compound  score  A34  measures  physical  interactions 
and  reflects  congruence  between  procedural  knowledge 
and  affordances  in  the  tool.  The  systematic  differences 
in  A34  indicate  that  the  task  was  more  easily  and 
efficiently  accomplished  with  version  C.  Again,  there 
were  no  differences  between  versions  A  and  B. 


CONCLUSIONS  AND  IMPLICATIONS  FOR 
BATTLESPACE  MANAGEMENT  SYSTEMS 

We  have  seen  that  an  information  organization  scheme 
based  on  procedural  knowledge  (Condition  C)  can 
facilitate  performance  at  a  complex,  time-driven  task. 
Moreover,  performance  without  a  reasoning-congruent 
information  scheme  hindered  performance,  regardless 
of  the  information  organization.  When  the  decision 
maker  must  expend  both  time  and  cognitive  resources 
to  compensate  for  the  tool,  those  resources  are  not 
available  to  perform  the  task.  Thus,  the  right 
organization  scheme  can  provide  the  affordances  for 
improved  cognitive  performance. 

How  would  this  work  in  a  Battlespace  Management 
System?  As  I  am  most  familiar  with  submarine  systems 
and  with  meteorological  systems  Til  use  one  of  those, 
the  submarine,  as  an  example.  The  submarine  systems 
include  multiple  sensor  performance  prediction 
algorithms,  sonar  sensors,  target-motion-analysis 
algorithms,  and  battlespace  displays.  These  tools 
correspond  to  the  functions  of  search,  detect,  track, 
classify,  localize,  etc.  However,  when  we  examine  the 
behavior  of  expert  submariners,  they  do  not  limit 
themselves  to  this  sequence  (Gray,  Kirschenbaum,  & 
Ehret,  1997;  Kirschenbaum,  1992).  They  iterate  among 
the  tools  as  they  employ  specific  information  gathering 
strategies.  Thus  a  Battlespace  Management  System  for 
submariners  might,  for  example,  facilitate  comparing 
the  output  of  a  target-motion-analysis  tool  to  the  sonar 
traces  at  that  bearing.  Would  the  proposed  course, 
speed,  and  range  actually  fall  within  the  sonar  trace,  as 
displayed?  How  well  would  it  match  the  region  where 
the  sonar  could  detect?  We  are  currently  building 
displays  to  answer  these  questions  by  showing  these 
detection  regions  in  3-D,  along  with  the  possible 
tracks.  Thus,  we  facilitate  the  very  comparison 
procedures  that  we  have  observed  submarine  decision 
makers  using. 

The  submarine  3-D  display  work  is  just  beginning.  The 
effects  on  performance  have  yet  to  be  tested.  The 
approach  is  much  like  that  used  in  the  experiment 
reported  above.  If  the  results  replicate  in  this  domain 
differences  in  affordances  for  procedural  knowledge 
will  again  support  differences  in  performance.  While 
information  systems  have  always  been  developed  by 
analyzing  perceived  needs,  a  radically  different 
suggestion  is  to  design  information  management 
decision  aids  the  way  new  walkways  are  sometimes 
planned.  That  is,  where  natural  paths  occur  because  of 
repeated  use  by  pedestrians,  constructed  (concrete, 
macadam,  gravel,  etc.)  pathways  are  built.  These  are 
not  planned  a  priori,  but  develop  from  use.  Only  after 
the  grass  has  been  worn,  are  constructed  paths  built.  In 
the  same  way,  the  information  organization  schemes 
should  provide  affordances  for  knowledge  in  the  head  - 
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-  the  procedural  paths  we  create  through  the  task  and 
the  information. 
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Summary:  New  information  technology  promises 
more  information  and  advanced  automated  functions  in 
future  cockpits  of  military  aircraft.  Flowever  the 
cognitive  human  capabilities  stay  the  same.  This  may 
result  in  an  overload  of  the  human  pilot.  Cognitive 
assistant  systems  are  being  developed  to  compensate 
for  this  mismatch.  This  paper  introduces  principals  of 
cognitive  systems  which  exhibit  human-like 
capabilities  as  interpretation  and  diagnosis  of  the 
situation,  planning  and  decision  making.  Furthermore, 
CAMA  (Crew  Assistant  Military  Aircraft),  a  prototype 
of  a  cognitive  assistant  system,  will  be  introduced. 
CAMA’s  functionality  will  be  shown  and  some  results 
from  flight  simulator  test  runs  will  be  presented. 

Motivation:  Environment  and  scenarios  of  military 
transport  missions  have  changed  over  the  last  few 
years  and  will  definitely  undergo  even  more  changes  in 
the  next  decade.  New  information  technology, 
including  telecommunication  as  well  as  hardware 
which  is  continuously  growing  more  powerful  will  find 
its  way  into  the  future  military  aircraft.  Online  data  of 
upcoming  threats,  detailed  weather  information,  terrain 
data  and  knowledge  about  weapon  systems  will  be 
available.  Combined  air  operations  with  participation 
of  AWACS,  fighters,  bombers  and  transport  aircraft  are 
likely  with  the  need  for  more  communication. 

There  is  a  rising  amount  of  mission-relevant 
information,  that  has  to  be  processed  by  the  human 
operator  who  is  also  in  charge  of  flying  the  aircraft. 
Considering  the  complexity  and  manifold  of 
automation  in  current  cockpits  and  even  more  in  those 
of  the  next  generation  it  can  clearly  be  seen,  that  it  will 
become  more  and  more  difficult  for  the  human  to  keep 
situation  awareness  and  perform  all  the  tasks  in  an 
efficient  way  without  errors. 

This  leads  to  the  central  question: 

How  can  we  make  the  best  use  of  the  potential  given  by 
the  new  technologies  without  overloading  the  cognitive 
capabilities  of  the  human  operator  ? 


There  is  an  approach  where  automation  in  the 
conventional  way  is  being  added  to  the  cockpit,  hoping 
for  increased  productivity  and  effectiveness.  As  we 
know  from  experience,  however  the  conventional 
automation  can  increase  safety  only  up  to  a  certain 
level.  Further  increase  of  complexity  in  the 
conventional  way  can  lead  to  a  safety  decline  as  shown 
in  figure  1  in  its  principal  relationships. 

Productivity  Safety 


Figure  1:  The  Effect  of  Conventional  and 
Cognitive  Automation  on  Productivity 
and  Safety 

Recent  accidents  of  commercial  aircraft  with  state-of- 
the-art  “conventional  cockpit  automation”  provided 
sufficient  evidence  for  this  particular  consequence.  [1] 
identifies  besides  complexity  as  such  also  other  design 
elements  more  or  less  as  part  of  complexity  like 
coupling  of  automated  features, 
autonomy  with  unexpected  self  initiated  machine 
behaviour  and 
inadequate  feedback 

which  are  typical  causes  for  respective  mishaps.  In 
military  aviation  the  situation  can  be  expected  to  be 
critical  due  to  permanently  increasing  requirements  for 
information  processing. 

Cognitive  automation:  How  can  this  situation  be  dealt 
with?  The  critical  point  is,  how  automation  can  be  done 
in  an  effective  manner.  Automation  should  not  be  a 
replacement  for  the  pilot,  but  instead  should  work  in  a 
cooperative  way  with  the  pilot.  In  the  ideal  case  it 
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should  work  like  a  kind  of  “electronic  crewmember”, 
with  the  cognitive  capabilities  like  those  of  the  human, 
but  without  all  its  possible  deficiencies. 

[2]  postulates  basic  design  requirements  founded  on 
these  cognitive  capabilities: 

Requirement  (1)  is  to  avoid  failings  in  situation 
awareness  and  reads: 

It  must  be  ensured  along  with  the  representation  of  the 
full  picture  of  the  flight  situation  that  the  attention  of 
the  cockpit  crew  is  guided  towards  the  objectively  most 
urgent  task  or  sub-task  as  demanded  in  that  situation. 

Requirement  (2)  is  to  avoid  overcharge  in  decision 
making/planning/plan  execution  and  reads: 

Situation  awareness  might  have  been  achieved  and  still 
a  situation  with  overcharge  of  the  cockpit  crew  might 
come  up.  In  this  case  the  situation  has  to  be 
automatically  transferred  into  a  situation  which  can  be 
handled  by  the  crew  in  a  normal  manner. 

“Cognitive  automation”  is  the  only  way  to  ensure 
increase  of  productivity  through  automation  without 
loss  of  safety  (see  fig.  1). 

The  difference  between  cognitive  and  conventional 
automation  can  also  be  illustrated  by  Rasmussen's 
scheme  of  human  cognitive  behaviour 

[3] . 


Figure  2:  Conventional  vs.  cognitive  Automation  [4] 


As  figure  2  shows,  conventional  automation  covers 
nearly  the  whole  of  skill  based  human  behaviour.  The 
rule  based  behaviour  can  only  partly  be  covered,  and 
on  the  knowledge  based  level  only  planning 
calculations  can  be  provided  by  conventional 
automation. 

Cognitive  automation  comprises  the  entire  rule  and 
knowledge  based  level,  as  well  as  the  skill  based  level, 
thereby  giving  the  system  human  like  capabilities  to: 

-  independently  assess  the  current  goals  of  the  crew, 
as  well  as  information  about  the  aircraft,  the 
environment  including  the  tactical  situation,  the 
weapon  systems  and  the  aircrew  activities 

-  understand  the  flight  situation  by  independently 
interpreting  the  situation  subject  to  the  goals 


-  detect  the  pilot's  intents  and  possible  errors 

-  detect  possible  conflicts  of  current  plans  but  also 
the  opportunities  arising  from  the  changing 
environment 

-  know  which  information  the  crew  needs 

-  support  necessary  re-planning  and  decision  making 

-  initiate  a  natural,  human- like  communication  to 
match  the  system's  internal  pictures  of  the  situation 
with  those  of  the  pilot. 

The  symbiosis  of  cognitive  automation  combined  with 
the  strength  of  the  human  will  lead  to  a  more  efficient 
and  safer  mission  execution 

The  Cognitive  Process:  To  realise  the  cognitive 
approach  as  a  technical  process  human  cognition 
provides  a  good  guideline.  The  following  core 
elements  can  be  identified: 

Situation  monitoring 
(perception  and  interpretation) 

Diagnosis  of  the  situation 

Decision  making  and/or  planning 
Execution/activation 

They  are  forming  the  cognitive  loop  as  shown  in  figure 
3.  The  environment  of  the  cognitive  assistant,  which  is 
named  the  real  world,  presents  stimuli,  which  can  be 
detected  by  different  kinds  of  sensory  systems.  Both 
the  environmental  stimuli  outside  and  inside  the 
cockpit  are  taken  into  account.  This  represents  the 
situation  monitoring  element,  which  comprises  the 
process  of  perception  of  all  relevant  situation  features. 
This  is  closely  interrelated  with  the  process  of  situation 
analysis  in  order  to  achieve  a  certain  level  of 
abstraction,  thereby  establishing  situation-relevant 
“objects”  which  help  to  understand  what  is  different 


between  the  expected  and  actual  situation.  These 
differences  are  dealt  with  in  a  higher  level  of 
abstraction  by  the  so-called  situation  diagnosis  process. 
The  differences  are  evaluated  against  given  objectives, 
the  relevant  goals ,  which  are  known  to  be  pursued 
during  the  mission,  and  which  are  the  same  the  aircrew 
has  in  mind.  Only  the  knowledge  about  these  goals 
makes  situation  awareness  possible  in  the  technical 
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cognitive  loop.  Thereby  conflicts  and/or  opportunities 
may  be  detected  which  may  call  for  immediate  actions 
or  some  flight  plan  changes. 

In  the  latter  case  a  planning  process  is  activated  to 
generate  alternatives  for  interim-goals,  plans,  and 
actions.  In  compliance  with  the  given  overall 
objectives,  the  most  appropriate  ones  are  chosen  for 
proposals. 

Concerning  the  assistant  system,  the  execution  element 
of  the  cognitive  process  plays  a  very  central  and 
important  role,  as  it  includes  the  communication  with 
the  crew.  It  is  carried  it  out  on  the  basis  of  profound 
internal  knowledge  about  what  information  the  aircrew 
is  looking  for,  why  and  when.  On  the  other  hand,  the 
crew  should  at  any  time  be  able  to  ask  for  certain 
information  within  the  system.  A  sophisticated  MMI  is 
required  to  accomplish  this  task. 

It  is  also  taken  into  consideration  that  the  crew  may 
react  different  compared  to  the  systems  proposals, 
because  certain  factors  were  not  taken  into 
consideration  at  system  design  time.  Thereby,  new 
stimuli  are  generated  for  the  cognitive  loop,  which 
starts  again  and  copes  with  the  crews  action.  The 
feedback  via  these  stimuli  creates  a  kind  of  implicit 
communication. 

CAMA  -  The  Prototype  of  a  Cognitive  Assistant 
System: 

Military  transport  missions  put  great  demands  on  the 
crew.  The  typical  scenario  is  composed  of  IFR  and 
tactical  flight  sections,  as  shown  in  figure  4. 


While  flying  IFR,  the  aircraft  may  operate  in  a  high 
density  airspace.  Separation  to  other  aircraft  has  to  be 
ensured.  The  tactical  scenario  is  entered  via  a  transition 
corridor.  Constraints  in  time  and  space  have  to  be  met. 
Tactical  flying  will  be  mostly  low  level  flying,  using 
terrain  masking,  even  under  adverse  weather 
conditions.  Additionally,  the  scenario  changes  at  a  high 
rate  along  with  quick  reaction  required  at  times. 
Concerning  these  conditions,  technical  cognitive 
assistance  for  the  flight  crew  seems  to  be  very 
promising. 

Therefore,  the  German  DoD  started  a  program,  called 
CAMA  (Crew  Assistant  Military  Aircraft),  in  order  to 
have  demonstrated  the  power  of  cognitive  automation 
for  transport  missions.  CAMA  as  a  prototype  cognitive 


assistant  system  has  been  developed  by  the  University 
of  German  Armed  Forces  Munich  in  close  cooperation 
with  DaimlerChrysler  Aerospace,  ESG 
(Elektroniksystem  und  Logistik  GmbH)  and  DLR 
(Deutsches  Zentrum  fur  Luff-  und  Raumfahrt)  (see  [7] 
[8]). 

Structure  and  Functionality  of  CAMA: 

The  Crew  Assistant  Military  Aircraft  provides 
functionalities  in  compliance  with  many  parts  of  the 
cognitive  loop.  Again  as  depicted  in  figure  5  the 
system  is  embedded  in  the  real  world  environment. 
Information  about  this  environment  can  be  perceived 
by  means  of  sensors  and  data  links. 

The  outer  layer  of  CAMA  performs  perception  and 
interpretation  of  the  relevant  situation  elements  of  the 
real  world.  The  process  of  environment  interpretation 
as  well  as  the  interpretation  of  the  aircraft  state 
provides  information  about  the  actual  weather,  the 
proximity  to  the  terrain,  other  aircraft,  as  well  as  the 
current  state  of  aircraft  subsystems.  Tactical 
information  which  consists  of  the  mission  task,  ingress 
and  egress  corridors  and  actual  threat  situation  may  be 
fed  into  the  system.  Additionally,  data  from  computer 
vision  systems  are  included  for  machine  perception  of 
relevant  obstacles  like  landing  strips  and  obstacles  on 
uncontrolled  strips  under  low  visibility  conditions. 

All  these  pieces  of  relevant  information  are  put 
together  to  form  a  central  situation  representation  that 
provides  all  the  data  which  other  CAMA  modules 
might  need  or  which  are  produced  for  further 
processing  like  the  evaluation  and  the  interpretation 
of  the  pilot’s  action.  This  core  element  of  CAMA 
forms  a  close  functional  relationship  with  the  inner 
functional  layer  of  the  system  for  diagnosis  and 
detection  of  conflicts  and  opportunities.  The  elements 
of  the  central  situation  representation  that  represent  the 
relevant  objects  of  the  real  world  are  evaluated  against 
the  expected  behaviour  of  the  pilot,  the  predicted  state 
of  the  aircraft  and  against  the  overall  mission 
objectives. 


In  order  to  monitor  the  pilot’s  behaviour  the  assistant 
system  needs  a  representation  of  the  expected  pilot 
actions.  In  CAMA  a  normative  model  [9]  describes  the 
pilot’s  behaviour  close  to  that  as  documented  in 
handbooks  and  air  traffic  regulations.  An  adaptive 
model  [10]  covers  behavioural  traits  of  the  individual 
pilot  flying.  If  the  actual  pilot  behaviour  differs  from 
the  internal  representations  of  CAMA  then  it  can  be 
classified  into  either  errors  or  intents  (see  [11]  [12]) 
This  classification  is  based  on  the  representation  of  the 
mission  objectives  and  flight  plan  goals.  These  can  be 
explicitly  stated  by  the  pilot  as  inputs  via  the  MMI  or 
can  be  implicitly  contained  in  the  pilot’s  intent  which  is 
continuously  monitored  by  CAMA. 

If  the  pilot  behaviour  is  classified  as  an  error  a  warning 
message  is  generated  and  a  corrective  action  is 
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proposed  to  the  pilot.  Upon  a  detected  intent  the 
internal  plan  is  adapted  accordingly.  Thus  an  implicit 
communication  between  the  pilot  and  the  system  takes 
place,  which  allows  the  pilot  to  react  to  the  current 
situation  without  having  to  tell  the  system  explicitly. 


derived  from  the  mission  order  (e.g.  entrance  corridors 
to  gaming  area,  drop-point,  time  over  target  etc.).  A 
Takeoff  to  landing’  mission  flight  plan  is  then 
generated.  The  IFR  flight  plan  as  part  of  it,  for 
example,  includes  the  lateral  flight  path  segments,  the 
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Figure  5:  Functional  structure  of  CAMA 


In  case  of  a  possible  traffic  conflict,  for  example, 
CAMA  detects  that  the  actual  behaviour  does  not 
comply  with  the  ‘safety’  objective  and  issues  visual 
and  acoustic  advice  as  part  of  the  Traffic  Alert  and 
Collision  Avoidance  Systems  (TCAS). 

Ground  proximity  is  continuously  monitored. 
Therefore,  all  possible  flight  trajectories,  achievable  by 
full  exploitation  of  the  aircraft  performance 
capabilities,  are  checked  for  terrain  avoidance  (using  a 
Digital  Elevation  Data  database).  Again  a  warning  is 
given,  visual  and  by  voice.  In  addition  an  evasive 
trajectory  is  generated. 

C  AMA  also  generates  proposals  for  acting  as  part  of 
the  conflict  resolution  which  involves  planning  and 
decision  making  support.  This  functionality  ranges 
from  very  short  term  planning  e.g.  collision  and  terrain 
avoidance  to  long  term  strategic  planning.  This  enables 
the  assistant  system  not  only  to  detect  the  possible 
conflict,  but  to  generate  a  conflict  solving  strategy. 
Again  all  relevant  data  needed  is  passed  over  from  the 
situation  representation  module.  In  case  of  overall 
flight  planning  all  accessible  information  about  the 
flight  is  passed  to  the  mission  planner.  This  includes 
mission  oriented  goals  and  constraints  that  can  be 


vertical  profile,  time  estimates  and  fuel  planning  [13] 
as  well  as  information  from  a  navigational  database. 
Mission  constraints  which  may  change  during  the  flight 
(e.g.  a  changed  exit  corridor  from  gaming  area)  or  ATC 
instructions  are  considered  during  the  planning  process. 
If  the  mission  order  leads  into  an  area  with  hostile 
radar  coverage,  the  Low  Altitude  planner  (see  [14] 
[15])  is  started  accordingly,  generating  a  minimum  risk 
route  with  a  maximum  probability  of  survival  in  a 
hostile  environment.  This  is  achieved  by  avoiding 
threatened  areas  if  possible,  minimizing  the  exposure 
to  unknown  threats  and  keeping  the  aircraft  clear  of 
terrain.  Therefore  the  mission  constraints,  the  tactical 
elements  and  the  resulting  threat  map,  the  terrain 
elevation  data  and  the  aircraft  performance  data  are  all 
taken  into  account.  The  generated  routes  are  passed  to 
the  crew  and  are  being  accepted  from  them  ,  modified 
or  rejected  respectively. 

The  calculation  is  done  in  terms  of  only  a  few  seconds, 
always  giving  the  pilot  an  idea  of  what  would  be  a 
good  plan  in  the  current  situation. 

The  advanced  functionality  of  CAMA  requires  a 
sophisticated  user  interface  to  let  the  pilot  make 
advantage  of  the  system  capabilities.  Care  has  to  be 
taken  in  the  design  of  the  MMI,  not  to  produce  an  extra 
cognitive  workload. 
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Figure  6:  CAMA  Nav-Display  with  Terrain  and 
Tactical  Elements 

The  Dialogue  Management  module  [18]  of  CAMA 
ensures  that  the  communication  is  provided  to  support 
situation  awareness  in  the  best  way  possible  in  all  flight 
conditions.  It  is  based  on  the  multimodal  approach, 
which  means  that  all  pilot  inputs  can  be  performed  by 
speech,  touch-sensitive  screens  as  well  as  conventional 
line  select  keys  or  switches.  Output  makes  use  of  the 
currently  available  display  technology  and  is  presented 
by  means  of  three  high  resolution  color  displays. 
Speech  output  is  used  in  parallel  to  textual  messages. 
The  simple  graphical  user  interface  delivers  a  good 
usability  already  after  a  short  introduction  to  the 
system. 

Pilot  inputs  can  be: 

Request  of  flight  planning  actions 
Activation,  modification  or  rejection  of  proposals 
Activation  of  actions  related  to  warnings 
Retrieval  of  information 
Autopilot  operations 
Configuration  of  the  MMI 
Radio  management 
CAMA  outputs  can  be: 

Presentation  of  calculated  flight  plan  proposals  in 
graphical  as  well  as  textual  form 
Situation  presentation  including  tactical  and  threat 
information 

Warnings  about  detected  conflicts 
Recommendation  about  explicit  actions 
Messages  in  reply  to  requests 
Acknowledgement  of  speech  input 
Presentation  of  complex  actions  like 
briefings,  checklists  etc. 

Several  MMI  devices  provide  support  for  the  flight 
guidance  task.  For  low  level  flying  under  difficult 
weather  conditions  the  primary  flight  display  can  be 
switched  to  a  3-dimensional  presentation  of  the 
surrounding  environment  [13]. 


Results  of  Simulator  Flight  Trials: 

CAMA  is  integrated  in  the  flight  simulator  of  the 
University  of  the  German  Armed  Forces,  Munich.  This 
simulator  provides  a  wide  field  of  view  visual 
simulation.  It  is  based  on  digital  terrain  and  feature 
data  and  shows  objects  like  rivers,  streets,  railroads  and 
powerlines  which  makes  it  suitable  for  low  level  flight 
simulations  based  on  terrestrial  navigation.  Three  high 
resolution  colour  monitors  with  touch-overlay  are  used 
to  display  CAMA  outputs.  Also  a  number  of  realistic 
flight  controls  are  available,  including  a  throttle  box, 
flaps,  gear  and  spoiler  levers,  as  well  as  an  Airbus-type 
flight  control  unit  for  autopilot  functions.  All  controls 
can  be  actively  driven  by  CAMA  on  request  of  the 
pilot. 


Figure  7:  Test  Flight  Simulator 


In  November  1997  and  May  1998  flight  simulator  test 
runs  were  conducted  (see  [17]).  10  German  Airforce 
transport  pilots  (Airlifter  Wing  61,  Landsberg)  were 
participating  as  test  subjects.  The  pilots  were  tasked 
with  full  scale  military  air  transport  missions.  This 
comprised  a  mission  briefing  with  following  takeoff 
from  base,  an  IFR  leg  to  the  ingress  corridor  and  a  low 
level  flight  to  a  drop  zone.  The  flight  over  hostile  area 
contained  a  dynamic  tactical  scenario  with  multiple 
SAM  stations  (Surface  to  Air  Missiles).  After  the  drop 
was  accomplished  the  flight  was  led  to  the  egress 
corridor,  followed  by  an  IFR  flight  segment  to  the 
home  base. 

Each  subject  had  to  perform  the  mission  three  times. 
There  was  not  much  time  needed  for  familiarisation 
and  training  on  the  system. 

To  set  up  an  realistic  level  of  workload  several  scenario 
items  were  put  in  the  missions,  which  required  an 
action  by  the  pilot. 

The  IFR  segment  incorporated: 

Adverse  weather  conditions 

High  density  airspace  (Other  aircraft  crossing  the 

own  flight  path) 

Changing  availability  of  landing  sites 
ATC  communication  (e.g.  clearances,  radar¬ 
vectoring,  redirection) 

The  tactical  segment  incorporated: 

Varying  SAM  sites 
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Drop  procedure 
Changed  egress  corridor 
Redirect  to  new  destination 

All  ratings  were  given  within  a  range  from  1  to  7, 
where  1  represented  the  best  and  7  the  worst  score.  A 
choice  of  the  results  is  shown  in  figure  8,  9  and  10, 
where  the  ratings  are  numbered  due  to  the  order  of  test 
runs. 

A  detailed  and  complete  documentation  of  the  test  runs 
and  its  results  is  given  in  [17]. 


an  assistant  system  like  CAM  A.  Like  the  test  subjects 
stated,  CAM  A: 

increases  Flight  Safety 
increases  Mission  Efficiency 
increases  Survival  Probability 

A  more  objective  analysis  of  the  flight  simulator  trials 
was  done  by  [20]  using  an  eye  tracking  system  and  a 
data  recording  tool.  More  information  on  this  topic  can 
be  found  in  the  respective  paper  in  the  same 
proceeding. 


(a)  /  always  understood  CAMA's  actions 


1  2  3  4  5  6  7 


(b)  I  was  (made)  aware  of  my  own  faults 
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Figure  8:  Evaluation  of  the  Cooperative  Approach 
ofCAMA 
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Figure  9:  Acceptance  of  CAMA  by  Pilots 


(a)  CAMA  increases  Flight  Safety 
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(b)  CAMA 


increases  Mission  Efficiency 
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(c)  CAMA  increases  Survival  Probability 
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Figure  10:  Overall  Evaluation  of  CAMA 


Especially  the  rating  concerning  the  overall  evalutaion 
as  shown  in  figure  10  points  out  clearly  the  benefits  of 


Actual  research: 


Figure  11:  Experimental  Aircraft  ATT  AS 


Recently  CAMA  was  being  integrated  in  the  in-flight 
simulator  ATTAS  of  the  DLR  (shown  in  figure  1 1)  and 
was  successfully  tested  and  demonstrated  in  several 
flight  experiments  in  March  2000.  Further  trials  are 
scheduled  for  November  2000.  These  flight  tests 
comprised  IFR  and  low  level  flight  segments  as  they 
occur  in  a  military  air  transport  mission.  Again  subjects 
were  experienced  air  transport  pilots  from  the  German 
Air  Force.  Data  from  sensor  input  as  well  as  the 
internal  system  states  were  recorded,  which  will  allow 
a  replay  of  the  conducted  flights  in  the  experimental 
flight  simulator  at  the  university  of  armed  forces  in 
Munich. 


Conclusion:  Future  battlefield  scenarios  will  be 
characterised  by  the  availability  of  a  greater  amount  of 
information.  Onboard  information  processing  puts 
great  demand  on  the  aircrew,  which  may  lead  to 
overcharging  of  the  crew. 

To  cope  with  these  changing  conditions,  the  approach 
of  a  cognitive  assistant  system  was  investigated.  It 
offers  support  to  the  aircrew  regarding  enhancement  of 
situation  awareness,  handling  of  multifunctional  tasks 
and  situation-dependent  balancing  of  workload  for  the 
sake  of  mission  effectiveness  and  safety.  It  has  become 
increasingly  evident  that  this  cannot  be  achieved 
without  moving  towards  the  cognitive  approach. 

The  presented  approach  and  its  realisation  in  the 
prototype  system  CAMA  has  been  demonstrated.  The 
benefits  are  already  demonstrated  in  the  course  of 
simulator  trials  and  In-flight  demonstrations. 
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Abstract 

Technology  pushes  for  sensor  and  weapon  systems  as 
well  as  for  command,  control,  communication,  and 
information  systems  have  increased  the  amount  and 
complexity  of  information  available  while  the  time  to 
process  that  information  has  dramatically  decreased. 
Additionally,  recent  changes  in  military  situations  and 
doctrines  have  given  rise  to  the  need  for  computer-based 
aids  that  can  support  human  operators  in  getting  situation 
awareness  and  reacting  to  novel  complex  and  rapidly 
changing  situations.  A  concept  has  been  developed  to 
support  the  members  of  the  decision  making  team  in 
combat  information  centers  of  the  German  Navy  vessels 
by  knowledge-based  user  interfaces.  Such  interfaces  will 
ease  the  burden  of  the  decision  makers  in  all  phases  of 
the  military  command  and  control  cycle  and  enhance  the 
effectiveness  of  the  decision  making  process  in  novel 
military  scenarios,  e.g.,  in  Littoral  Warfare,  Crisis  and 
Low  Intensity  Conflicts,  or  Missions  other  than  War. 
The  paper  starts  with  a  general  problem  description  and 
a  framework  of  operator  support  possibilities  based  on  a 
hierarchical  structure  of  human  task  performance  with 
different  levels  of  situational  complexity.  It  follows  the 
description  of  a  generic  support  concept  by  means  of 
knowledge-based  user  interfaces  consisting  of  a 
knowledge-based  assistance  system  and  an  interactive 
multimedia  user  interface.  Finally,  as  an  example  the 
implementation  of  the  conceptual  work  into  a 
demonstrator  of  operator  support  in  naval  anti-air 
warfare  situations  is  presented.  With  this  demonstrator 
the  effectiveness  of  decision  making  and  action  taking 
support  by  a  knowledge-based  user  interface  coul^l 
shown. 

1  Introduction 

Technology  pushes  for  sensor  and  weapon  systems  as 
well  as  for  all  kinds  of  military  command  and  control 
systems  (C2/C3/C4I)  have  increased  the  amount  and 
complexity  of  information  at  hand  while  the  time 
available  to  process  that  information  has  dramatically 
decreased. 


Additionally,  in  actual  military  operations,  e.g.,  in 
Littoral  Warfare,  Crisis  and  Low  Intensity  Conflicts,  or 
Missions  other  than  War,  operators  who  are  responsible 
for  planning  and  decision  making  are  faced  with  natural 
dynamic  situations  which  are  characterized  by  extremely 
rapid  changes  in  the  tactical  situation,  highly  uncertain 
information,  and  a  large  variety  of  potential  situational 
hypotheses.  These  decision  makers  undergo  high  mental 
stress  due  to  the  need  to  respond  quickly  and  accurately, 
or  face  potentially  fatal  consequences. 

It  may  currently  not  be  possible  to  design  a  system 
which  can  cope  with  all  conceivable  events  in  highly 
ambiguous  situations,  for  instance,  with  those  found  in 
novel  military  operations.  But  it  is  already  possible  to 
develop  a  system  that  complements  human’s  abilities  in 
perceiving  and  assessing  such  situations  as  well  as 
responding  appropriate  in  unknown  situations. 

Operator  support  by  intelligent  and  adaptive  knowledge- 
based  user  interfaces  is  considered  to  be  a  viable 
approach  to  overcome  some  of  those  difficulties 
decision-makers  are  faced  with  when  having  to  cope 
with  complex  command  and  control  systems  in  novel 
military  situations.  Such  user  interfaces  consist  of  a 
knowledge-based  assistance  system  and  an  interactive 
graphical  or  multimedia  user  interface.  They  can  support 
military  decision  makers  in  performing  information 
gathering,  information  processing,  and  information 
entering  in  all  phases  of  a  command  and  control  (C2) 
cycle,  i.e.,  in  situation  perception  (observe),  situation 
assessment  (orient),  decision  making  (decide),  and  action 
taking  (act). 

2  Human  Task  Performance  and 
Operator  Support  in  Complex 
Situations 

Situational  awareness  (SA),  that  is  a  reliable  assessment 
of  the  situation  in  which  a  ship  operates,  is  vital  for  the 
successful  completion  of  its  mission.  To  establish  and 
maintain  SA,  information  from  own  sensors  or  other 
sources  of  significant  data  and  conditions  must  be 
processed.  This  information  concerns  the  tactical 
environment  as  well  as  the  own  combat  (sub)system(s). 
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Figure  1 :  Situation  awareness  and  decision  making  with  uncertainty  and  time  pressure 


In  order  to  provide  a  reliable  information  basis  for 
carrying  out  missions,  it  will  be  necessary  to  assess  and 
reassess  the  situation  on  a  continuous  basis.  Normally, 
little  is  known  about  the  operational  environment  with 
certainty.  Therefore,  to  establish  SA  hypotheses  about 
the  behavior  of  both  identified  and  unidentified  objects 
as  well  as  about  the  overall  tactical  and  strategic 
situation  have  to  be  generated.  Uncertainty  and 
imprecision  in  observation  and  information  gathering 
will  quickly  result  in  very  large  numbers  of  hypotheses 
that  will  have  to  be  managed  by  human  decision  makers. 
This  demand  cannot  adequately  be  satisfied  under 
constraints  of  information  overload  and  time  pressure 
(Fig.  1). 

When  being  confronted  with  complex  situations  three 
levels  (styles)  of  decision  making  can  be  defined  as 
reference  points  on  a  "cognitive  continuum"  [Amat, 
1995]: 

•  reactive  level  (panic  style)  - 

in  the  case  that  SA  may  be  incomplete  or  even  very 
reduced  because  of  time  pressure  and  information 
uncertainty  the  decision  making  style  will  correspond 
to  a  panic  behavior  that  leads  to  impulsive  selection 
of  a  course  of  action  based  on  a  primary  situation 
hypothesis. 

•  tactical  level  (holistic  style)  - 

familiarization  and  the  expertise  acquired  play  a 
crucial  role  in  operator  decision  making.  When  the 
operator  recognizes  a  situation  as  belonging  to  his 
catalogue  of  experienced  situations,  he  associates  to 
this  picture  a  course  of  action.  The  first  hypothesis 
that  works  is  adopted  and  implemented.  This  holistic 
style  of  decision  usually  provides  good  results.  Under 
high  time  pressure  or  mental  workload  most  of  the 
time  holistic  strategies  will  also  be  used  with  good 
but  sometimes  with  less  desirable  results. 

•  strategic  level  (analytical  style)  - 

the  analytical  style  corresponds  to  a  "theoretical" 
strategy.  It  is  the  most  time  consuming  and  mentally 


demanding  style.  On  this  level  an  operator  makes  a 
detailed  assessment  of  a  situation,  gathers  the 
maximum  of  data,  defines  the  problem,  forms  a  list 
of  alternate  solutions  to  the  problem,  chooses 
selection  criteria,  ranks  them  by  priority  and  selects 
the  alternative  having  the  greatest  weight  within  the 
space  defined  by  the  selection  criteria.  The  aim  of 
this  strategy  is  to  find  the  optimum  solution. 

The  task  of  gathering  and  correctly  combining  different 
types  of  data,  information,  reports,  and  messages  and 
then  drawing  accurate  conclusions  still  remains  the 
responsibility  of  the  military  decision  makers  who  are 
usually  under  great  strain  during  tactical  operations. 
Generic  cognitive  tasks  to  be  performed  by  means  of  C2 
systems  are  information  gathering/situation  perception, 
situation  assessment,  goal  establishment,  decision 
making/action  planning,  action  command,  and  control  of 
action  accomplishment  for  goal  achievement.  These 
cognitive  tasks  describe  the  course  of  activities  in 
military  C2  cycles  and  in  nonmilitary  decision  situations, 
too.  Differences  in  situational  familiarity,  information 
uncertainty  and  time  pressure  are  related  to  different 
cognitive  processes  corresponding  with  differences  in 
control  effort  and  time  consumption.  Effort  and  time 
increase  from  simple  decisions  with  quick  reactions  for 
routine  tasks  in  clear  situations  to  task  of  higher 
complexity  with  cognitive  demanding  tasks  in 
ambiguous  situations. 

For  developing  support  systems  that  assist  the  operator 
in  complex  decision  situations  it  is  useful  to  obtain  and 
apply  a  model  of  the  human  decision  making  and 
problem  solving  process.  This  model  has  not  to  be  an 
exact  reproduction  of  human  cognitive  processes. 
Rather,  it  should  enable  identifying  possibilities  for 
situation-,  task-,  and  user-based  assistance  of  the 
decision  making  and  problem  solving  process.  Such 
assistance  should  comprise  information  presentation  for 
situation  assessment,  information  processing  for  solution 
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Figure  2:  Concept  structure  of  human  problem-solving  activities 


preparation,  and  operator  guidance  for  information 
entering  to  execute  tasks. 

To  establish  such  a  general  model,  complex  operator 
tasks  are  divided  into  three  different  performance  phases 
according  to  the  normal  approach  of  operator  problem 
solving  behavior:  (1)  situation  perception,  (2) 
situation/problem  assessment,  and  (3)  solution 
generation.  Additionally  to  structuring  problem  solving 
tasks  horizontally  into  phases,  a  vertical  structure  seems 
to  be  appropriate  for  each  of  the  three  performance 
phases.  This  vertical  structure  is  related  to  the  specific 
types  of  situations  with  different  cognitive  complexity 
one  has  to  cope  with  in  natural  settings.  For  this  structure 
the  conceptual  model  of  Rasmussen  [1983]  for  skill-, 
rule-,  and  knowledge-based  operator  behavior  in 
performing  complex  tasks  can  be  adapted  as  a 
framework.  The  resulting  concept  takes  into  account  the 
different  situations  which  arise,  for  instance,  in  novel 
military  scenarios,  i.e.,  routine,  familiar,  and  unfamiliar 
situations.  The  hierarchical  differentiation  dependent  on 
situational  familiarity,  expertise,  and  cognitive  operator 
demand  corresponds  also  to  the  steps  of  mental  activity 
which  Lim  et  al.  [1996]  applied  for  describing  human- 
computer  interactions.  This  model  contains  the  following 
steps:  perception,  interpretation,  evaluation,  goals  and 
intention,  action  specification,  and  execution.  Combining 
the  hierarchical  differentiation  of  the  three  performance 
phases  of  problem-solving  tasks  with  the  cognitive  steps 
of  the  model  of  Lim  et  al.  a  conceptual  framework  has 
been  developed  that  describes  the  problem  space  of 
human  decision-making  and  problem-solving  in  complex 
naturalistic  situations  with  a  3  x  3  matrix  (Fig.  2). 

On  the  lowest  level  of  this  framework  skill-based 
behavior  corresponds  to  nearly  unconsciously  and 
automatic  information  processing  in  routine  situations  or 
under  time  pressure  and  to  executing  tasks  at  an  reactive 
level  (Fig.  1).  On  the  second  level  rule-based  behavior 
corresponds  to  stereotyped  information  processing  in 
well-known  situations  at  an  tactical  level  or  holistic 
style.  On  the  upper  level  knowledge-based  behavior 
corresponds  to  a  conscious,  controlled,  and  analytical 


problem  solving  using  situation  analysis  and  evaluation 
as  well  as  planning  and  decision  making  activities. 

Applying  the  hierarchical  structure  of  human  problem 
solving  activities  to  a  realistic  problem  situation,  the 
following  steps  of  information  processing,  i.e.,  situation 
perception,  situation  assessment,  and  solution  generation 
can  be  identified: 

•  information,  i.e.,  states  from  environmental  objects, 
the  technical  system  or  the  operator  behavior  is 
perceived  and  monitored  by  a  situation  assessment 
process.  Relevant  status  changes,  i.e.,  events  are 
detected  and  the  related  information  processed.  The 
result  leads  directly  to  a  reflexive  action  by  the 
decision/action  step  of  the  solution  generation 
process  if  the  information  belongs  to  a  well  known 
and  recognized  routine  situation. 

•  if  the  available  information  does  not  directly  lead  to 
a  reflexive  situation/reaction  routine  the  situation 
assessment  process  has  to  interpret  the  available 
information  using  known  heuristics,  i.e.,  using 
already  existing  hypotheses  about  the  familiar 
situation,  to  come  to  a  situation  interpretation.  The 
result  of  this  interpretation  will  be  used  by  the  action 
planning  step  of  the  solution  generation  process  to 
decide  about  the  appropriate  reaction  or  to  plan  the 
appropriate  course  of  actions,  if  necessary. 

•  if  the  information  is  ambiguous  or  uncertain  and  the 
situation  is  unfamiliar  or  ill  structured  so  that  the 
situation  assessment  process  is  unable  to  directly 
recognize  or  interpret  the  situation  by  known  rules,  a 
situation  diagnosis  has  to  be  performed.  On  the  basis 
of  the  diagnosis  a  course  of  actions  will  be  identified 
by  the  action  planning  step  of  the  solution 
generation  process  if  the  result  of  the  diagnosis 
corresponds  to  the  predefined  goal.  Otherwise,  a 
new  goal  and  the  corresponding  action  sequence  for 
achieving  it  has  to  be  determined  by  the  solution 
generation  process.  This  decision  depends  on  the 
complexity  of  the  hypothesis  determined  by  the 
situation  assessment  process. 
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3  Concept  for  a  knowledge-based 
operator  support 

To  support  the  human  operator  in  SA  and  decision 
making  with  complex  systems  in  naturalistic  situations 
adaptive  aiding  concepts  have  been  developed  [Rouse  et 
al.,  1988;  Rouse,  1991],  These  concepts  have  been 
developed  to  overcome  human  deficiencies  in 
information  perception  and  processing.  They  have 
actually  been  applied  as  aids  for  aircraft  pilots  as  a  so- 
called  ’’pilot’s  associate"  [Rouse  et  al.,  1990;  Amalberti 
et  al.,  1992;  Wittig,  et  al.,  1992)].  The  basic  idea  of  these 
concepts  is  that  an  overall  automation  must  not  be  the 
objective  of  system  development  [Bainbridge,  1987]. 
The  human  operator  should  be  involved  in  the  decision 
making  process  as  far  as  his  abilities  and  his 
performance  are  sufficient  for  achieving  mission  goals. 
An  aid  is  provided  only  to  enhance  human  abilities  (e.g. 
in  detecting  and  evaluating  complex  patterns  or  reacting 
on  unforeseen  events)  and  to  overcome  human 
deficiencies  (e.g.  when  doing  mathematical 
calculations),  i.e.,  to  complement  individual  human 
performances. 

3.1  Concept  of  a  Knowledge-Based  Assistant 

For  the  reasons  mentioned,  the  concept  of  human- 
centered  automation  recommends  a  computerized 
assistant  that  complements  the  operator  like  a  human 
partner.  The  human  user  is  engaged  in  a  cooperative 
process  in  which  human  and  computer  assistant  both 
initiate  communication,  monitor  events  and  perform 
tasks.  The  computer  assistant  does  not  act  as  an  interface 
or  layer  between  the  user  and  the  command  system.  In 
fact,  the  most  successful  assistant  systems  are  those  that 
do  not  prohibit  the  user  from  taking  actions  and  fulfilling 
tasks  personally,  i.e.,  behaving  as  a  personal  assistant 
that  cooperates  with  the  user  on  the  same  task.  Thus,  in 
parallel  to  the  human  operator,  the  assistant  monitors  the 


situation  (e.g.  states  of  the  system  and  the  environment) 
and,  additionally,  operator  actions.  If  the  assistant 
discovers  critical  situations  or  inappropriate  operator 
behavior,  it  may  automatically  perform  some  operator- 
related  functions.  Faulty  behavior  of  the  operator  will  be 
classified,  announced,  and  if  there  is  no  reaction  from 
the  operator,  possibly  compensated  by  the  assistant.  But 
in  any  case  the  user  is  able  to  bypass  the  assistant,  so  that 
the  responsibility  and  ultimate  decision  remains  with  the 
human  operator. 

The  knowledge-based  user  interface  [Berheide  et  al., 
1995]  to  support  human  operators  consists  of  a 
knowledge-based  user  assistant  (KBUA)  and  an 
interactive  multimedia  user  interface  (Fig.  3). 
Information  presentation  and  the  user  dialog  with  the  C3 
system  are  accomplished  via  the  user  interface  which 
acts  as  the  communication  tool  for  the  user  with  assistant 
and  C3  system. 

The  knowledge-based  user  assistant  is  not  an  automation 
or  expert  system  in  the  conventional  sense  but  contains 
the  knowledge  of  domain  experts  and  makes  it  available 
on  the  user  interface  to  assist  the  human  operator,  e.g., 
according  to  situation,  mission,  task,  system  states,  or 
operator  needs.  This  aiding  or  assistance  will  be  attained 
by  a  situation-  and  task-related  information  presentation 
and  information  processing,  as  well  as  by  means  of 
operator  action  guidance  according  to  situation  relevant 
tasks  and  course  of  actions  combined  with  information 
input. 

3.2  Functional  Structure  of  the  Knowledge-Based 
User  Assistant 

The  functional  structure  of  the  KBUA  itself  consists  of 
three  components,  i.e.,  a  situation  monitor,  a  solution 
generator,  and  an  information  manager  (Fig.  4), 
representing  the  three  different  areas  of  the  general  task 
performance  or  support  structure:  (2)  situation/problem 
assessment,  (3)  solution  generation,  and  (1)  situation 
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Figure  3:  Concept  of  a  knowledge-based  user  interface 
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Figure  4:  Functional  structure  of  the  knowledge-based  user  assistant 


perception/information  presentation  [Dorfel  et  al.,  1997], 

The  situation  monitor  supports  the  operator  in  gathering 
information  and  assessing  situations  by  means  of 
reviewing  and  analyzing  situational  data  with  regard  to 
relevant  situational  events.  The  situation  monitor  itself 
consist  of  three  different  parts  monitoring  the  external 
situation,  the  overall  system  status,  and  the  operator 
behavior  by  means  of  predefined  filter  functions.  The 
present  implementation  of  this  monitor  contains  only 
filter  functions  to  monitor  the  external  situation.  Later  on 
it  will  contain  functions  to  monitor  system  status  and 
operator  actions,  too. 

Problem-  and  task-relevant  situation  assessment  by  the 
situation  monitor  is  the  basis  for  the  solution  generator 
to  define  goals,  plan  actions  to  reach  newly  defined  goals 
or  predefined  objectives,  and  generate  appropriate 
decision  proposals.  Dependent  on  situation  assessment 
and  the  time  available  the  planning  part  of  the  solution 
generator  decides  about  function  allocation  between 
human  operator  and  machine  system  components,  i.e., 
"which  function"  to  be  accomplished  "by  whom"  and 
"when",  as  well  as  about  the  information  and  action 
requirements  of  the  human  operator,  i.e.,  "what" 
information  or  action  possibility  to  provide  and  "when". 

Necessary  information  as  well  as  required  action 
possibilities  are  presented  on  the  user  interface  by  means 
of  the  information  manager ,  supporting  the  information 
perception  process  of  the  human  operator  by  deciding 
"how"  the  presentation  should  be  designed  and 
performed. 

This  general  concept  of  the  knowledge-based  assistant 
can  be  applied  to  different  problem  areas  as  well  as  to 
different  kinds  of  operator  support.  However,  specific 
problem  areas  require  establishing  domain-specific 
contents  for  data  and  knowledge  bases  as  well  as 
availability  of  problem-specific  reasoning  and  problem 
solving  processes,  like  rule-based  or  case-based 
reasoning,  diagnosis,  and  handling  of  uncertainty  by 
probabilistic  reasoning  or  belief  networks.  Using  the 
proposed  3x3  matrix  of  problem  solving  performance 
(Fig.  2),  as  an  orientation  already  existing  solution 


systems  for  restrictively  supporting  reactive,  planning,  or 
decision  making  tasks  can  be  identified  and  integrated 
advantageously  into  a  new  support  concept.  In  this  way, 
the  matrix  structure  with  its  elements  allows  a  modular 
development  and  a  stepwise  implementation  of  a 
knowledge-based  support  concept.  This  enables 
developers  to  quickly  react  on  situational  demands. 

4  Application  of  the  Proposed 
Concepts 

To  demonstrate  the  applicability  of  the  described  KBUA 
concept  for  efficiently  supporting  military  decision 
makers  in  complex  situations  demonstrator  systems  have 
been  prototypically  developed  for  exemplary  tasks.  In 
the  following  one  application  will  be  presented  in  some 
detail. 

In  a  research  project  for  the  German  Navy  the  concept  of 
a  knowledge-based  user  interface  has  been  applied  to 
develop  an  aid  for  the  Identification/Recognition 
(ID/REC)  process  in  ship  air  defense  [Dorfel  et  al., 
1999].  The  special  support  concept  consists  of  a  number 
of  support  functions  for  an  event-related  information  and 
task  management  to  efficiently  perform  the  identification 
process. 

The  development  started  with  the  specification  of  a 
realistic  crisis  reaction  scenario.  On  the  basis  of  this 
scenario  the  following  operator  tasks  have  been 
identified  for  support:  a)  monitoring  the  established  air 
picture  to  detect  specific  task  relevant  events,  b) 
identifying  newly  detected  tracks,  c)  changing  the 
identity  of  already  identified  tracks  if  necessary,  and  d) 
performing  the  investigate  procedure  for  SUSPECT  or 
HOSTILE  tracks  showing  threat  relevant  behavior. 

In  cooperation  with  naval  experts  a  limited  number  of 
exemplary  criteria  have  been  defined  from  the  scenario 
and  assigned  to  ID  states  of  a  track.  From  these  criteria 
an  event  list  has  been  deduced  to  define  ID  state 
transitions  caused  by  situational  events.  Fig.  5  shows  the 
corresponding  state  transition  diagram  of  a  track 
comprising  ID  states,  exemplary  situational  events,  and 
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Figure  5:  ID  state  transition  diagram  for  exemplary  events 


related  state  transitions.  This  diagram  has  been  used  for 
developing  the  logic  of  the  situation  monitor,  the 
functions  of  the  solution  generator,  and  the  user  interface 
configurations  applied  by  the  information  manager. 

The  ID/REC  process  has  been  supported  by  presenting 
actual  data  as  well  as  preprocessed  situational  and  task 
relevant  information  on  an  interactive  graphical  user 
interface.  Additionally,  the  operator  has  been  guided  by 
providing  information  entering  possibilities  on  this 
interface  (Fig.  6). 

Each  task-relevant  situational  event  will  be  notified  at 
the  user  interface  by  presenting  a  virtual  event/action 
button  (VEAB)  and  a  related  information/action 
window.  The  VEABs  are  prompts  or  warnings  about  the 


occurrence  of  task-relevant  events  or  calls  on 
performance  of  special  tasks  for  related  tracks, 
respectively.  Additionally,  they  present  important 
information  about  those  tracks.  The  VEABs  are  arranged 
by  priority  where  VEABs  for  critical  events  or  tasks  with 
a  high  performance  priority  are  placed  most  left.  After 
confirming  a  notified  event  or  accomplishing  a  task 
called  for  the  related  VEAB  will  be  deleted. 

Activating  a  VEAB  will  open  the  related 
information/action  window  (I AW)  for  the  specific  track. 
Track  number  and  specific  icon  as  well  as  type  and  class 
of  the  track  are  presented.  Situation  perception  as  well  as 
situation  assessment  are  supported  by  the  presentation  of 
actual  track  attribute  values  and  preprocessed 
information  like  trend  information,  min/max  values  of 
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Figure  6:  Graphical  interactive  user  interface  for  ID/REC 
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the  kinematic  track  data  during  the  observation  period, 
histories  of  speed,  altitude,  and  distance  as  well  as 
relations  of  altitude  versus  distance  or  speed.  Besides 
these  quantitative  information  additional  qualitative 
information  is  presented  textually.  The  sum  of  this 
information  serves  as  an  exhaustive  operator  support  in 
situation/problem  awareness. 

Additional  support  for  situation  perception  and 
assessment  as  well  as  for  decision  making  and  task 
performance  is  given  by  the  presentation  of  task  relevant 
events  and  related  actions  in  an  event/action  list.  This  list 
contains  events  occurred  combined  with  relevant  action 
proposals  in  a  chronological  order.  Selecting  a  special 
event  from  the  list  will  show  additional  information 
characterizing  and  explaining  that  event.  Selecting  a 
proposed  action  from  the  list  will  show  additional 
information  related  to  the  causing  event  as  well  as  a 
decision  or  action  suggestion  with  explanation  for  that 
special  suggestion.  These  decision/action  suggestions 
together  with  the  presentation  of  alternatives  support  the 
solution  generation  of  human  operators. 

In  action  execution  the  human  operator  is  supported  in 
the  way  that  he  is  able  to  directly  confirm  the  system’s 
suggestion,  choose  and  execute  an  proposed 
decision/action  alternative  or  delete  the  decision/action 
suggestion.  Additionally,  the  execution  of  actions 
consisting  of  an  activity  sequence  is  supported  by  the 
presentation  of  a  time-line  of  the  activities  to  be 
performed  as  well  as  of  specific  alerts  to  perform  these 
activities  optimally  in  time  and  distance  or  space. 

Besides  the  situation-relevant  information  presented 
automatically  by  the  support  system  there  is  the 
possibility  to  present  additional  preprocessed 
information  on  operator  demand.  This  are  histories  of  the 
kinematic  data  altitude,  distance,  and  speed  as  well  as 
relations  between  these  values. 

For  operator  relief  there  is  the  possibility  on  operator 
demand  to  automatically  identify  tracks  fulfilling 
predefined  criteria  like,  e.g.,  flying  en  route  an  air  lane 
and  emitting  a  civil  IFF  code. 

5  Summary  and  Conclusion 

Difficulties  of  human  operators  performing  demanding 
problem  solving  tasks  in  natural  settings  have  given  rise 
to  the  need  for  support  systems  that  can  assist  them  in 
assessing  and  reacting  to  complex  and  rapidly  changing 
situations.  Obviously,  problem  solving  tasks  in  such 
situations,  especially  military  situations,  may  be 
supported  by  knowledge-based  systems.  At  present  it 
may  not  yet  be  possible  to  design  a  system  addressing  all 
possible  events  in  highly  ambiguous  situations,  such  as 
those  found,  for  instance,  in  military  crisis  reaction 
operations  or  operations  other  than  war.  But  it  is  already 
possible  to  develop  support  systems  that  complement 
human’s  ability  in  perceiving  such  situations  and  in 
responding  appropriately  in  novel  situations.  The 
concepts  presented  in  this  paper  provide  a  systematic 


approach  for  the  design  of  knowledge-based  systems  to 
support  problem  solving  in  complex  situations.  The  basis 
of  the  approach  is  a  generic  description  and  framework 
of  the  human  problem  solving  process  itself. 

The  knowledge-based  user  assistant  system  developed 
and  implemented  for  supporting  decision  making  and 
action  taking  in  naval  air  defense  situations  demonstrates 
the  potential  of  such  an  approach.  With  the  developed 
interface  demonstrator,  the  advantages  of  this  concept 
for  operator  support  in  complex  problem  situations  like 
identification/recognition  (ID/REC)  could  be  shown. 

The  information  presentation  provides  the  decision 
makers  with  needed  information,  thus  supporting  all 
three  levels  of  SA:  perception,  comprehension,  and 
projection  in  the  tactical  and  operational  area.  The 
information  is  displayed  corresponding  to  the  task¬ 
relevant  operator  needs,  i.e.,  he  can  easily  perceive  and 
use  the  information  without  undue  cognitive  effort. 
Thus,  operator  workload  is  reduced  because  the 
information  corresponds  directly  to  his  task  goal,  i.e.,  for 
identification  of  air  targets  to  encounter  possible  threats 
in  air  defense. 

In  addition  to  SA  support  through  prompts  and  warnings 
operators  are  alerted  to  task-relevant  events  and 
prompted  to  the  appropriate  actions  by  specific  proposals 
and  by  guidance  for  action  sequences.  In  addition  to 
situation  awareness,  this  improves  decision  making  and 
action  command.  However,  the  ultimate  decision 
remains  with  the  human  operator  himself,  thus,  keeping 
him  as  an  integral  and  critical  part  in  the  decision  loop. 

The  knowledge-based  user  interface  concept  developed 
is  a  very  general  one  and  can  be  applied  to  different 
kinds  of  operator  support  systems  for  a  variety  of 
different  missions  and  tasks.  Human  operators  will  be 
supported  by  information  presentation  and  user  guidance 
which  are  adapted  to  mission,  situation,  task,  system 
status,  as  well  as  to  operator  abilities.  For  the 
development  of  the  demonstrator  an  object-oriented 
approach  and  a  modular  architecture  have  been  applied 
that  allow  changes  and  extensions  of  the  demonstrator  to 
be  made  easily.  Therefore  new  as  well  as  additional 
functions  can  be  integrated  quickly. 

The  concept  of  the  knowledge-based  user  interface 
consisting  of  a  knowledge-based  user  assistant  (KBUA) 
and  a  multi-media  user  interface  will  be  further 
developed.  It  is  actually  adopted  for  developing  a 
support  system  to  aid  the  CIC  team  in  developing  and 
using  doctrines  for  automatically  controlling  the  combat 
direction  system  of  a  German  Navy  frigate. 
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Summary 

In  this  paper  we  provide  an  overview  of  our  research 
investigating  what  functionality  should  be  provided  to 
users  of  a  future  Joint  Battlespace  Infosphere  (JBI).  We 
characterize  and  discuss  the  development  of  the  JBI  as  a 
new  form  of  automation  that  employs  intelligent  agents 
to  autonomously  seek,  retrieve,  and  fuse  information  . 
We  believe  that  the  development  of  new  types  of  direct 
manipulation  interfaces  are  the  best  approach  to 
achieving  JBI  goals  of  reducing  decision  time  and 
manning  while  maintaining  positive  control  over  the 
command  and  control  (C2)  system.  Further,  we  argue 
that  the  integration  of  direct  manipulation  interface 
techniques  with  interface  agents  will  change  the  HCI 
from  a  mechanism  to  execute  tasks  into  a  decision-aid 
that  supports  cognitive  information  processing.  We 
contextualize  this  discussion  by  providing  an  overview  of 
the  Air  Force  Research  Laboratory’s  Human  Interaction 
with  Software  Agents  (HISA)  project.  This  effort  is 
developing  a  HCI  for  Air  Mobility  Command’s  (AMC) 
Tanker  Airlift  Control  Center  (TACC)  that  interacts  with 
operational  C2  systems  through  intelligent  agents,  similar 
to  the  manner  of  the  proposed  JBI. 

I.  The  Joint  Battlespace  Infosphere  Concept 
(JBI)  as  a  Three-Layer  Model 

To  achieve  the  goal  of  information  dominance  on  the 
battlefield,  the  U.S.  Air  Force  is  exploring  the 
development  of  a  Joint  Battlespace  Infosphere  (JBI).  The 
JBI  is  proposed  as  an  integrated  C2  system  encompassing 
the  entirety  of  Air  Force  operations.  The  JBI  will  be 


implemented  through  data  and  communications  networks 
which  will  enable  warfighters  to  plug  into  the  JBI  in  a 
fashion  analogous  to  logging  onto  the  Internet.  The  JBI 
will  then  provide  these  warfighters  access  to  the  complete 
range  of  information  products  and  services  necessary  for 
operational  decision  making.  As  a  comprehensive 
decision  making  environment,  the  JBI  would  serve  as 
both  the  repository  and  generator  of  mission  critical  data. 
Individual  warfighters  would  provide  data  for  the  JBI  and 
in  turn  receive  fused  presentations  of  information  tailored 
to  their  goals  and  needs. 

Two  of  the  primary  goals  of  the  JBI  concept  are  the 
reduction  of  warfighter  decision  times  and  staffing 
demands.  The  first  goal  is  to  be  obtained  by  more 
efficiently  accessing  and  fusing  decision-critical  data  and 
more  effectively  presenting  it  for  employment  in  decision 
making  tasks.  The  second  goal  is  to  be  obtained  by 
eliminating  inefficiencies  and  redundancies  in  the  current 
‘stove-piped’  information  architectures  among  numerous 
operational  units.  Both  goals  can  be  obtained  to  the 
extent  the  cognitive  burdens  of  decision  makers  are 
reduced.  Reducing  decision  makers’  cognitive  burdens 
serves  the  first  goal  by  facilitating  the  decision  processes 
of  any  given  decision  maker.  It  serves  the  second  goal  by 
leveraging  decision  making  efficiency  to  permit  smaller 
staffs  to  equal  or  exceed  current  performance  standards. 

Implementation  of  the  JBI  requires  translating  the 
concept  into  deployable  decision  support  tools  and 
systems.  Our  analysis  of  the  JBI  concept  suggests  that 
the  requisite  network-oriented  deployable  products  will 
evidence  three  types  of  functionality,  which  in  turn  can 
be  described  in  terms  of  three  layers: 
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•  A  network  services  layer  which  provides 
connectivity  between  the  various  C2  systems; 

•  an  application  services  layer  (ASL)  which  provides 
services  such  as  planning,  scheduling,  and 
information  fusion,  often  mediated  through 
intelligent  agents;  and 

•  a  human  computer  interface  (HCI)  layer  through 
which  warfighters  receive  information  and  enact 
operational  tasks. 

II.  Agent  Support  Criteria  for  the  HCI  and  ASL 
Layers 

The  primary  areas  for  intelligent  agent  support  will  be  the 
ASL  and  HCI  layers,  and  it  will  be  these  two  layers  upon 
which  our  discussion  will  focus.  It  is  important  to  clearly 
distinguish  between  these  two  layers,  primarily  because 
their  respective  implementations  will  employ  intelligent 
agents,  albeit  quite  differently  (Milewski  &  Lewis, 
1997).  Although  both  involve  multiple  processes  (i.e., 
agents)  communicating  with  one  another  in  an  intelligent 
fashion,  our  project  experience  suggests  it  is  critical  to 
‘frame’  the  purpose  of  these  layers  in  distinct  ways.  This 
‘framing’  allows  for  more  precise  identification  of 
agents’  roles,  as  well  as  illuminating  the  optimum 
referential  context  for  design  and  implementation  of 
agent  support,  for  each  layer. 

To  illustrate,  in  the  following  paragraphs  we  shall 
summarize  the  ASL  and  HCI  layers  with  respect  to  three 
critical  dimensions  of  agent  implementation: 

•  ontology  -  the  fundamental  semantics  underlying 
terms  of  reference  and  types  of  inference. 

•  homo-/heterogeneity  -  the  differential  unity  / 
multiplicity  of  elements  (or  element  types)  engaged 
by  users,  the  agents,  or  both. 

•  autonomy  -  the  degree  to  which  a  given  agent  (or 
class  of  agents)  functions  outside  the  scope  of  user 
monitoring  and/or  direction. 

1 1. A.  The  ASL  Layer 

The  ASL  is  best  characterized  as  a  functional  architecture 
designed  to  accomplish  specific  tasks  such  as  scheduling 
aircraft  and  crews  for  specific  missions  or  planning  the 
movement  of  forces  into  a  theater  of  operations.  Given 
the  diversity  in  both  legacy  and  prospective  mission- 
critical  systems,  the  ASL  must  provide  for  a  flexible 
mapping  of  tasks  to  computers  (or  computer  processes). 
In  a  traditional  C2  system  a  task  (e.g.,  scheduling)  is 
often  accomplished  by  a  specific  computer  running  a 
specific  (scheduling)  application  program.  In  contrast,  in 
the  JBI  tasks  will  not  be  mapped  to  specific  hardware  / 
software  platforms.  Agents  working  for  a  specific  user 
will  request  a  service,  and  other  agents  will  manage  the 
details  of  how  and  where  that  service  is  actually 


accomplished  (e.g.,  on  which  computer;  using  which 
software).  This  illustrates  two  important  points.  First, 
the  ‘ontology’  for  ASL  agents  must  emphasize 
procedural  logic,  support  systems,  data  routing  protocols, 
etc.  Second,  the  multiplicity  of  items  (and  item  types) 
referenced  in  this  ASL  ontology  means  that  heterogeneity 
of  functionalities  (and  loci  of  functionalities)  will  be  a 
key  concern  in  ASL  design  and  implementation. 

This  heterogeneity  extends  to  the  agents  themselves. 
That  is,  the  ASL  layer  will  not  exhibit  a  single  standard 
language,  type  of  agent,  or  form  of  agent  communication. 
Instead,  there  will  be  a  variety  of  agents  that  speak  a 
diversity  of  languages  (e.g.,  Knowledge  Query  Mark-up 
Language  [KQML]  {Finn,  Labrou,  &  Mayfield,  1997}, 
Knowledgeable  Agent-oriented  System 

[KaoS]{  Bradshaw,  Deutfield,  Benoit,  &  Wolley,  1997}) 
and  employ  a  number  of  communication  protocols.  This 
conglomerate  of  distributed  disparate  agents  will 
advertise  and  broker  services  among  themselves  in  order 
to  find  the  optimal  means  to  complete  a  specific  task  in 
the  operational  context  of  priorities,  situational 
constraints,  and  other  related  tasks  underway. 

Agents’  ‘brokerage’  of  diverse  goals,  tasks,  conditions, 
and  functionalities  will  add  value  to  the  extent  that  it 
manages  the  relevant  complexity  (i.e.,  complexity  of  type 
-  heterogeneity)  on  behalf  of  (e.g.)  planners  and 
commanders.  In  this  case,  the  obvious  tactic  for 
complexity  management  is  to  allow  the  agents  to 
automatically  handle  the  details  of  user-defined  tasks. 
Phrased  another  way,  ASL  agents’  value  will  be  directly 
proportional  to  the  amount  of  detailed  tasking  they  can 
accomplish  without  users’  direct  inspection  and 
guidance.  As  such,  the  hallmark  of  useful  ASL  agents  is 
capacity  for  autonomous  action  (vis  a  vis  the  user). 

II.B.  The  HCI  Layer 

In  contrast,  the  HCI  layer  is  defined  with  respect  to  the 
user  him/herself.  In  a  traditional  C2  system,  most  tasks 
are  initiated  and  managed  by  a  user.  In  contrast,  design 
goals  for  the  JBI  include  reducing  the  decision-making 
cycle  time,  while  simultaneously  reducing  the  number  of 
personnel,  and  while  maintaining  positive  (human) 
control  over  the  weapon  systems.  The  HCI  layer,  then, 
must  provide  the  capacity  for  user  inspection  of  task 
parameters  as  well  as  the  means  through  which  the  user 
invokes  and  manages  his/her  tasks.  In  contrast  to  the 
ASL  layer,  the  HCI  layer  is  best  characterized  as  an 
architecture  of  forms  (as  opposed  to  functions)  designed 
to  facilitate  understanding  of  and  control  over  specific 
tasks.  More  specifically,  it  is  implemented  as  a 
collection  of  graphical  user  interface  [GUI]  widgets  that 
actualize  a  user-interface  model  in  which  the  user 
delegates  to  and  collaborates  with  intelligent  software 
agents. 
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The  HCI  layer  is  effective  to  the  extent  it  facilitates  non- 
autonomous  actions  -  i.e.,  those  actions  reserved  for 
direct  human  control.  This  fact  underlies  one  critical 
distinction  between  design  approaches  to  the  ASL  and 
HCI  layers.  In  the  ASL  layer,  the  opportunity  for  agent 
autonomy  affords  designers  the  ability  to  design  with 
respect  to  any  functionalities  the  software  agents  can 
implement.  In  the  HCI  layer,  the  requirement  for 
discretionary  user  control  (i.e.,  HCI  agent  non-autonomy) 
forces  designers  to  constrain  themselves  to  prioritizing 
those  particular  functionalities  the  human  can  and/or 
must  manage. 

The  most  obvious  thing  the  human  must  manage  is  his  / 
her  interactivity  with  the  HCI  layer  itself.  As  the 
primary  point  of  engagement  between  user  and  system, 
the  HCI  layer  is  the  explicit  ‘point  of  service’  for  the  JBI. 
For  the  JBI  to  support  effective  work  and  decision 
aiding,  the  HCI  layer  must  itself  be  designed  as  an 
effective  work  /  decision  aid.  This  means  that  the  HCI 
layer  must  be  designed  so  as  to  reflect  key  referential  and 
operational  aspects  of  the  task  and  related  decision 
space(s).  As  such,  the  HCI  layer  must  be  based  on  an 
ontology  consistent  with  the  user’s  viewpoint  -  i.e.,  an 
ontology  focusing  upon  the  mission,  specific  tasks,  and 
decisions. 

The  kind  of  diversity  (heterogeneity)  that  is  usefully 
exploitable  in  the  ASL  layer  is  itself  a  serious  problem 
for  humans  to  routinely  handle.  The  user’s  cognitive 
workload  should  not  be  increased  by  forcing  him/her  to 
deal  with  details  of  the  HCI  layer’s  implementation  - 


e.g.,  the  specifics  of  how  the  HCI  agents  interact  with  the 
ASL  agents.  This  means  the  HCI  layer  must  emphasize 
both  simplicity  (non-complexity)  and  consistency  of  both 
form  and  function  in  portraying  and  addressing  tasks, 
conditions,  tools,  and  functionalities.  As  such, 
homogeneity  must  rule  in  any  HCI.  That  is,  a  user  must 
be  able  to  rely  upon  a  GUI  widget  coherently  displaying 
task  parameters  and  consistently  performing  any 
functions  he/she  invokes  in  response.  If  a  widget 
displays  the  status  of  an  airbase  in  one  fashion  at  one 
time,  the  user  should  expect  it  to  portray  that  status  in  the 
same  fashion  (a)  for  other  airbases  anytime  and/or  (b) 
that  same  airbase  some  other  time.  Similarly,  if  a  widget 
allows  specific  actions  (e.g.,  drilldown  to  more  detailed 
status  data)  on  one  occasion,  the  same  functionality 
should  be  predictable  the  next  time  the  widget  is  used. 

II.  C.  Summary  of  ASL  /  HCI  Design  Tradeoffs 

Figure  1  is  offered  as  a  summary  illustration  of  these 
points.  The  ‘squares’  represent  interface  elements  visible 
to  the  user,  and  the  ‘spheres’  represent  the  software 
agents  servicing  the  interface  as  well  as  accomplishing 
the  ASL  layer  functions. 

At  the  interface,  the  HCI  layer  offers  specific 
functionality  (e.g.,  scheduling  missions,  requesting 
resupply,  etc.).  The  user  may  not  know  (or  care)  that  the 
functionality  being  provided  is  mediated  through 
interface  agents  (the  spheres  clustered  behind  each 
interface  element).  He/she  may  not  know  because  the 
interface  reflects  a  task-oriented  (as  opposed  to  a  system- 
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Figure  1:  The  HCI  and  ASL  layers 
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oriented)  ontology.  He/she  may  not  be  aware  that  as  we 
move  farther  into  the  ASL  layer,  agents  increasingly 
interact  autonomously  among  themselves,  brokering 
services  and  accomplishing  tasks.  Conversely,  as  we 
move  in  the  other  direction  (from  ASL  toward  HCI), 
agents  (and/or  agent  sets)  increasingly  reflect  task- 
oriented  factors,  and  may  be  provided  interface  elements 
by  which  the  user  can  address  them  directly  (e.g.,  setting 
functional  preferences). 

In  between  the  extremes  of  the  ASL  and  HCI  deployment 
styles  is  the  gray  area  depicted  in  Figure  1 .  This  is  the 
domain  where  the  homogenous  HCI  layer's  agents 
interact  with  the  heterogeneous  agent-based  ASL 
architectures.  It  is  also  the  point  at  which  full  agent 
autonomy  comes  into  play.  Because  it  represents  the 
extent  of  HCI  service  ‘coverage’,  this  middle  ground 
demarcates  the  user’s  zone  of  awareness  with  respect  to 
the  JBI.  On  the  one  hand,  it  is  from  this  point  rightward 
(cf.  Figure  1)  that  autonomous  agents  can  be  trusted  to 
operate  out  of  sight  of  the  user.  On  the  other  hand,  it  is 
from  this  point  leftward  that  agents  are  usefully  made 
‘visible’  for  user  inspection  and  ‘available’  for  user 
manipulation.  For  example,  users  may  want  to  “drill- 
down”  into  this  area  to  tailor  agent  behavior  -  e.g., 
selecting  a  specific  module  for  a  particular  task  or 
creating  new  agents  to  serve  as  sentinels  for 
problematical  conditions.  As  such,  it  is  in  this  middle 
ground  that  tradeoffs  must  be  determined  involving  our 
two  critical  factors  of  homo-/heterogeneity  and 
autonomy. 

Perhaps  even  more  importantly,  this  middle  ground  is  the 
point  at  which  the  ASL  and  HCI  layers’  ontologies  must 
intersect  and  interoperate.  With  respect  to  Figure  1,  it  is 
from  this  point  rightward  that  the  ASL  layer’s  system- 
oriented  ontology  may  prevail,  and  it  is  from  this  point 
leftward  that  the  HCI  layer’s  task-oriented  ontology  must 
prevail.  For  the  above-cited  drill  down  to  be  effective, 
there  must  be  semantic  interoperability  between  the 
users’  conceptual  work  domain  model  (via  the  analogous 
HCI  layer  task  ontology)  and  the  ontologies  of  the  ASL 
agents  brokering  services  among  planners,  schedulers, 
search  engines,  etc. 

The  most  difficult  challenges  facing  JBI  developers 
concern  interoperability  tradeoffs  between  the  HCI  and 
ASL  layers  as  described  above.  They  must  provide  for 
interoperability  between  the  users’  homogenous  interface 
and  the  heterogeneous  agent  world.  This  functional 
interoperability  must  be  qualified  with  respect  to 
reasonable  allocation  of  users’  control  versus  agent 
autonomy.  Finally,  the  distinct  semantic  priorities  of  the 
ASL  and  HCI  layers  must  be  interwoven  through 
interoperability  of  their  respective  ontologies.  These 
difficult  goals  must  be  achieved  in  a  manner  that  (a) 
maximizes  functionality  provided  the  users;  (b) 
minimizes  users’  cognitive  workload;  (c)  maximizes 


system  operational  efficiency;  and  (d)  promotes  task 
effectiveness  in  JBI  applications. 

III.  Our  Work-Centered  Interface  Concept 

In  this  section  we  review  some  of  the  design  goals  we 
hope  to  achieve  by  creating  a  separate  HCI  layer,  where 
the  user’s  interaction  with  the  system  is  mediated  through 
interface  agents.  First,  we  outline  what  we  see  as  the 
primary  problem  -  cognitive  burdens  on  the  decision 
maker  entailed  in  addressing  two  distinct  ontologies 
(domains  of  reference  and  knowledge).  Second,  we 
identify  two  major  design  goals  for  alleviating  this 
problem.  Finally,  we  present  the  HISA  design  criteria 
through  which  we  pursued  these  design  goals. 

Ill  A.  The  Problem:  Complexity  of  Ontological 
Reference  in  User  /  System  Interaction 

In  general,  every  computer-implemented  decision  aid  can 
be  differentiated  into  two  distinct  functional  components: 

•  The  decision-making  component  supports  task- 
specific  decision  making  (e.g.,  deconflicting  an 
aircraft  scheduling  problem.  ).  The  decision¬ 
making  component  must  be  configured  so  as  to  allow 
the  user  to  address  the  task  he/she  is  executing. 

•  The  information  manipulation  component  supports 
task-specific  data  /  information  activities  (e.g., 
accessing  a  system  to  retrieve  data  or  to  assign  a  new 
mission  start  time.  The  information  manipulation 
component  must  be  configured  so  as  to  allow  the 
user  to  address  the  tools  (information  systems) 
he/she  employs  in  the  course  of  the  task. 

Owing  to  this  dichotomy  of  reference,  these  two  decision 
aid  components  differ  in  the  types  of  knowledge  that 
must  be  active.  A  decision-making  task  requires 
activation  of  a  task  domain  ontology  -  i.e.,  the  set  of 
specialized  terms,  meanings  and  relations  between  terms 
that  captures  or  represents  the  subject  matter  itself  (i.e., 
the  domain  knowledge  of  scheduling  goals  and 
constraints).  Information  manipulation  requires  activation 
of  a  system  ontology  -  i.e.,  the  set  of  specialized  terms, 
meanings  and  relations  between  terms  that  captures  or 
represents  working  knowledge  of  the  subject  matter 
documentation. 

Consequently,  a  user  engaged  in  decision-making  must 
engage  in  multi-tasking  behavior  which  involves 
(potentially  extensive)  shifting  between  the  frames  of 
references  (or  activate  ontologies)  of  the  systems  and 
task.  Let  us  illustrate  this  with  an  example.  To 
deconflict  a  scheduling  clash,  a  mission  planner  may 
have  to  access  two  or  more  systems.  Interacting  with 
each  system  requires  the  planner  to  develop  and  execute 
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an  information  manipulation  strategy.  This  may  require 
several  procedural  steps  such  as  logging  on  to  a  system, 
accessing  the  appropriate  data  base,  and  then  executing  a 
query.  Phrased  another  way,  the  user  must  generate  and 
work  through  a  plan  distinct  from,  but  potentially  of 
similar  complexity  as,  the  mission  plans  involved  in  the 
scheduling  clash.  In  utilizing  the  retrieved  data  for 
deconfliction,  the  planner  must  then  turn  to  an  entirely 
separate  referential  framework  reflecting  (e.g.)  aircraft, 
airfields,  and  planning  constraints.  In  other  words,  the 
planner  must  invoke  and  refer  to  a  referential  set  distinct 
from,  but  potentially  of  similar  complexity  as,  the  data 
dictionaries  underlying  the  retrieved  data. 

The  user  must  therefore  grapple  with  developing  a  single 
problem  solution  via  engagement  with  two  distinct 
referential  and  procedural  frameworks.  This  increases 
the  user’s  work  demands,  cognitive  burdens,  and  risk  of 
error.  With  respect  to  our  deconfliction  example,  the 
user  encounters  transcription  costs  in  assembling  relevant 
data  from  multiple  sources  (e.g.,  writing  down  or  printing 
out  conflicted  missions’  data  as  it  comes  in).  Once  all  the 
relevant  data  is  at  hand,  the  user  must  then  endure  the 
interpretation  costs  for  interrelating  a  set  of  data  field 
entries  and  a  set  of  mission  arrivals  /  departures  at  the 
given  airfield. 

III.B.  The  Solution :  Minimizing  Ontological 
Complexities  to  Reduce  User  Cognitive  Complexity 

The  above-cited  costs  are  a  matter  of  cognitive 
complexity.  Cognitive  complexity  (Chechile,  Eggleston, 
Fleischman,  &  Sasseville,  1989)  is  a  measure  of  how 
much  cognitive  resources  are  required  to  execute  an 
activity.  Note  that  cognitive  complexity  for  an  activity  is 
an  aggregate  of  complexity  of  the  information 
manipulation  and  decision-making  components. 
Cognitive  complexity  for  an  information  manipulation 
task  is  usually  a  function  of  how  much  planning  is 
required  to  execute  a  task.  Cognitive  complexity  for  a 
decision-making  task  is  harder  to  quantify  because  of  the 
variability  of  the  types  of  tasks  the  actor  is  engaged  in 
and  the  role  the  actor’s  skill  level  plays  in  task 
performance. 

This  dilemma  would  be  minimized  to  the  extent  the 
system  and  task  ontologies  correspond  (e.g.,  in 
terminology).  Unfortunately,  this  correspondence  is 
rarely  evident  in  management  information  systems. 
Moreover,  the  increasingly  integrated  network  character 
of  emerging  command  and  control  architectures  are 
predicated  upon  the  ability  of  warfighters  to  ‘drill  down’ 
(into  their  own  data  assets)  and  ‘reach  back’  (for  data 
assets  possessed  by  someone  else).  Because  the  trend  is 
toward  increased  referential  qualifications  (drill  down)  or 
more  numerous  data  sources  (reachback),  the  above-cited 


ontological  dilemma  can  only  become  more 
problematical. 

Much  of  the  HISA  interface  design  effort  was  directed 
toward  enhancing  task  /  system  correspondence  and 
reducing  mission  planners’  current  reliance  on  work¬ 
around  strategies  and  tools  (e.g.,  pen  and  paper).  Our 
design  goals  for  the  agent- based  HCI  layer  focused  upon 
providing  mission  planners  with  more  direct  support  for 
their  decision-making  through: 

•  increasing  the  time  the  user  operates  “on-task”  -  i.e., 
accomplishes  task  activities  by  working  with 
reference  to  the  task  domain. 

•  reducing  the  amount  of  time  the  user  digresses  “off- 
task”  -  i.e.,  is  sidetracked  into  activities  requiring 
reference  to  the  system  domain. 

III.C .  Our  Approach:  A  'Work  Centered1  Interface 
Style 

The  two  key  solution  criteria  cited  above  must  be 
reflected  in  design  and  development  work  to  obtain  the 
expected  payoffs.  To  accomplish  this,  we  translated  the 
goals  and  principles  cited  above  into  a  set  of  HISA 
interface  design  criteria  to  guide  our  work.  These  criteria 
reflect  the  following  priorities: 

•  maximize  explicit  reference  to  task  domain  elements 
in  the  on-screen  HISA  information  displays 

•  maximize  cross-reference  among  HISA  information 
displays  with  respect  to  core  task  domain  concepts 
(e.g.,  missions,  airfields,  courses  of  action) 

•  minimize  procedural  costs  for  accessing  and 
retrieving  relevant  data  (e.g.,  by  automating  queries) 

•  maximize  effective  fusion  of  data  from  the  multiple 
databases  with  which  the  planners  must  currently 
interact  (e.g.,  by  assembling  a  single  airfield 
summary  view  from  data  scattered  across  numerous 
database  tables) 

•  minimize  cognitive  burdens  for  identifying,  seeking, 
and/or  interpreting  relevant  information  (e.g.,  by 
reducing  interpretational  demands) 

The  implementation  strategy  uniting  the  above-cited 
design  goals  and  approaches  entailed  a  trade-off  between 
(a)  the  interfaces  engaged  by  the  users  and  (b)  the 
functionalities  delegated  to  the  software  agents.  We 
strove  to  configure  the  display  components  to  prioritize 
task  domain  referentiality,  and  we  prioritized  allocating 
system  domain-oriented  actions  (e.g.,  database  access)  to 
the  agents. 

This  is  not  simply  a  matter  of  providing  a  highly 
graphical  direct  manipulation  user  interface.  Direct 
manipulation  capabilities  do  help  reduce  the  cognitive 
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complexity  of  the  information  manipulation  component 
by  making  it  easier  to  directly  manipulate  information 
elements.  In  the  eventual  realization  of  the  JBI  concept, 
warfighters  must  access  multiple  systems,  each  one  of 
which  may  provide  a  very  different  interface.  Direct 
manipulation,  accordingly,  does  not  necessarily  eliminate 
the  need  to  switch  ontologies.  As  a  result,  direct 
manipulation  alone  is  not  sufficient  to  accomplish  our 
design  criteria  and  hence  our  design  goals. 

Interface  concepts  can  be  characterized  with  respect  to 
either  the  perspective  of  the  user  (e.g.,  direct 
manipulation)  or  the  perspective  of  the  system(s)  (e.g., 
object  oriented,  agent-based).  We  believe  the  key 
innovations  of  our  HISA  effort,  though  involving  novel 
system  capabilities,  are  best  characterized  from  the  user 
perspective.  Though  the  HISA  interface  elements  (as 
viewed  by  the  user)  certainly  represent  ‘direct 
manipulation’,  this  label  does  not  convey  what  we  see  as 
the  really  innovative  aspect  of  this  work.  Our  HISA 
interface  concepts  direct  as  much  of  this  direct 
manipulation  as  possible  to  task  (as  opposed  to  system) 
elements. 

In  other  words,  we  are  attempting  to  more  directly 
support  task  decision-making  by  effecting  a  closer 
correspondence  between  on-screen  display  elements  and 
elements  of  the  task  domain  (as  opposed  to  elements  of 
the  information  space).  In  effect,  we  are  making  the 
system  more  ‘transparent’  vis  a  vis  the  mission  planners’ 
tasks.  This  strategy  reduces  the  cognitive  complexity 
involved  in  addressing  task  activities  by  reducing  the 
procedural  and  interpretational  overhead  for  addressing 
task  issues  through  the  ‘lens’  of  support  system-specific 
interfaces.  This  allows  us  to  maximize  the  time  the  user 
spends  oriented  to  the  task  domain  itself  by  maximizing 
his/her  ability  to  address  task  activities  in  terms  of  task 
(as  opposed  to  system)  ontology.  We  call  an  interface 
which  realizes  our  design  criteria  work  centered. 

IV.  Our  HISA  Products  as  Work  Centered  Interfaces 

The  Air  Force  Research  Laboratory  (AFRL)  has  been 
researching  work  centered  interfaces  as  part  of  the 
“Human  Interaction  with  Software  Agents”  (HISA) 
project.  The  target  worksite  for  HISA  products  is  the 
Tanker  Airlift  Command  Center  (TACC)  -  a  mission 
planning  and  execution  center  within  the  USAF  Air 
Mobility  Command  (AMC).  TACC  units  plan,  schedule, 
and  monitor  airlift  missions  on  a  continuous  basis.  More 
particularly,  HISA  has  concentrated  on  the  specific 
category  of  channel  missions  -  those  missions  which  are 
routinely  conducted  along  established  routes.  Key 
characteristics  of  the  USAF  channel  mission  planning 
work  include: 

•  Long  lead  times  for  mission  plans .  Channel  mission 
plans  are  typically  drafted  and  published  months 


ahead  of  time  (in  advance  of  mission  take-off)  to 
enable  organizations  to  plan  family  moves. 

•  Heterogeneous  data  assets.  Missions  are  initially 
planned  using  one  system,  with  the  final  version 
being  ‘published’  to  another.  These  system  differ  in 
the  way  the  data  tables  are  laid  out.  Further, 
multiple  other  databases  each  uniquely  contain 
relevant  information  such  as  (e.g.)  airfield 
restrictions  and  alerts  on  airfield  status. 

•  High  cognitive  burden  for  data  access.  The 
multiplicity  and  diversity  of  record  tables  make  it 
laborious  to  track  down  specific  details  of  a  mission. 
When  obtaining  such  details  require  access  to 
multiple  tables  and/or  other  databases,  planners  must 
execute  multiple  queries  -  potentially  involving 
multiple  search  syntaxes. 

•  No  capacity  for  unified  issue  visualization.  The 
scheduling  system  provides  only  structured  textual 
records  of  mission  data,  arranged  by  mission.  To 
review  issues  involving  multiple  missions,  planners 
must  often  execute  a  query,  print  out  the  results,  and 
review  this  printout  manually.  Discerning  on-ground 
conflicts  at  a  given  airfield  typically  requires 
interrelating  mission  text  entries  by  drawing  lines 
among  them  with  a  pen. 

•  Little  or  no  automated  decision  support.  The  system 
provides  no  automated  inference  to  detect  conflicts 
among  mission  plans  as  they  are  accreted. 
Moreover,  the  system  provides  no  automated  alerts 
on  conflicts  and  other  problematical  conditions. 

•  Discontinuous  situation  awareness.  Once  a  channel 
mission  is  published  (months  ahead  of  time), 
conflicts  resulting  from  subsequently-published 
missions  can  go  undetected  (and  hence  unresolved) 
until  it  is  nearly  time  to  launch  the  mission. 

•  High  potential  for  time-critical  problem  solving 
under  duress.  In  accordance  with  TACC  business 
rules  and  policies,  channel  planners  must  usually 
defer  to  planners  of  other  missions  types  (e.g., 
contingency  missions)  when  resources  (e.g.,  aircraft) 
are  insufficient  to  execute  all  plans  at  once.  The 
above-cited  conditions  make  for  frequent  last-minute 
replanning  problems,  while  the  channel  missions’ 
low  prioritization  diminishes  planners’  ability  to 
definitively  resolve  those  problems  on  their  sole 
initiative. 

As  of  the  time  of  this  writing,  the  HISA  effort  had 
produced  design  specifications  for  a  work-oriented 
planner  interface,  as  well  as  dynamic  demonstration 
models  for  some  core  elements  of  this  interface.  Our 
HISA  interface  elements  have  been  demonstrated  in  real¬ 
time  interoperability  with  networked  data  sources, 
providing  concise  and  coherent  displays  of  mission 
planning  parameters  as  well  as  offering  proactive  support 
(e.g.,  alerts;  plan  conflict  data)  for  decision  makers.  The 
following  sections  offer  selected  examples  of  our  HISA 
interface  elements  and  illustrate  how  they  both  (a) 
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address  problems  faced  by  the  client  TACC  channel 
mission  planners  and  (b)  illustrate  the  principles,  goals, 
and  criteria  outlined  earlier  in  this  paper. 

IV A.  The  Foundation  for  Work-Oriented  Interface 
Design:  A  Task  Ontology 

Agent-based  support  will  afford  us  the  ability  to  shift  the 
users’  ‘field  of  vision’  from  the  machine  to  the  task  itself. 
The  ‘intelligence’  of  ASL  and  HCI  agents  will  relieve 
users  of  cognitive  burdens  attributable  to  having  to 
understand  the  mechanics  of  the  support  system  to  get  a 
task  accomplished.  As  a  result,  an  agent-based  HCI  layer 
allows  an  unprecedented  ability  to  reflect  the  ontology  of 
the  task  rather  than  the  ontology  of  the  system(s).  By 
disengaging  the  task  semantics  from  the  tool  semantics, 
we  have  been  able  to  design  our  HISA  HCI  layer 
elements  to  directly  reflect  the  mission  parameters 
comprising  the  critical  issues  in  the  planning  process,  as 
opposed  to  the  planning  artifacts  (e.g.,  cryptic  database 
records)  reflecting  the  limitations  of  the  planning 
documentation  (Eggleston,  1993). 

The  first  step  in  accomplishing  this  required  the 
development  of  a  coherent  task  ontology  which  was 
consistent  with  the  key  referential,  inferential,  and 
procedural  elements  by  which  users  engage  their  work.  It 
was  obvious  from  the  start  that  the  primary  object  of  task 
engagement  was  the  mission  plan  -  e.g.,  the  documented 
record  of  a  scheduled  mission  as  stored  in  GDSS. 
However,  it  was  equally  obvious  that  the  problematical 
issues  listed  above  all  related  to  grappling  with  this 
mission  plan  documentation  at  the  expense  of  efficiently 
and  effectively  addressing  the  subject  matter 
documented.  Our  first  goal  was  to  identify  the  key 
subject  matter  on  the  way  to  configuring  the  HISA 
interfaces  to  reflect  it. 

The  initial  knowledge  acquisition  efforts  clearly 
indicated  the  primary  object  of  referential  and  inferential 
engagement  was  the  mission  itself  —  i.e.,  the  act  of 
employing  an  aircraft  and  crew  to  transport  a  specific  set 
of  items  from  one  airfield  to  another.  We  therefore 
nominated  “mission”  to  be  the  core  construct  around 
which  to  develop  the  mission  planner  task  ontology. 
Further  analysis  (e.g.,  of  actual  and  representative 
problem  scenarios)  resulted  in  our  subdividing  this  core 
construct  into  three  components: 

•  Port  -  Either  one  of  the  airfields  involved  in  a  given 
mission  leg. 

•  En  Route  -  The  passage  of  the  loaded  aircraft  from 
one  Port  to  the  other. 


•  Package  -  The  aircraft,  crew,  load,  and  other  items 
required  to  perform  a  mission  leg. 

Our  early  knowledge  acquisition  indicated  that  problems 
were  typically  delimited  with  respect  to  one  or  another  of 
these  components.  For  example,  lack  of  a  functional 
aircraft  was  a  Package  issue.  Similarly,  weather- 
motivated  diversion  to  an  alternate  landing  site  was  an  En 
Route  issue,  and  exceeding  the  established  Maximum  On- 
Ground  (MOG)  limit  for  a  given  airfield  was  a  Port  issue. 
This  conceptual  model  allowed  the  HISA  team  to  create 
a  taxonomy  of  interface  displays  reflecting  both  (a)  a 
logical  taxonomy  of  subcomponents  of  the  core 
referential  construct  (i.e.,  the  mission),  as  well  as  (b)  a 
reasonable  categorization  of  known  task  problem 
features.  Identification  of  the  critical  data  and 
information  necessary  to  portray  each  of  these 
subcomponents  led  to  the  development  of  specialized 
displays  (termed  “Viewers”)  for  each.  One  such  display 
(the  “Port  Viewer”)  is  described  in  more  detail  later. 

This  initial  task  ontology  development  set  the  stage  for 
meeting  our  design  goals  of  prioritizing  “on  task”  user 
engagement.  More  specifically,  this  effort  allowed  us  to 
satisfy  our  design  criteria  of  maximizing  explicit 
reference  to  task  domain  elements;  maximizing  cross- 
reference  among  HISA  information  displays  with  respect 
to  core  task  domain  concepts;  and  minimizing  cognitive 
complexity  in  terms  of  interpretational  demands. 

IV.B.  Work-Oriented  Interface  Implementation: 
The  Port  Viewer 

The  best-received  of  our  work-oriented  displays  is  the 
‘Port  Viewer’  illustrated  below  in  Figure  2.  The  Port 
Viewer  is  a  discrete  interface  element  portraying  the 
arrival  and  departure  of  flights  for  a  given  airfield  for  a 
given  24-hour  period.  This  affords  direct  graphical 
summarization  of  conditions  which  planners  must 
currently  infer  from  a  large  text  printout.  By  portraying 
the  on-ground  circumstances  in  one  way  at  one  time,  we 
can  allow  agents  to  infer  and  depict  problematical 
conditions  (e.g.,  red  highlighting  of  the  period  during 
which  too  many  aircraft  are  present).  In  addition  to 
displaying  mission-critical  information,  the  Port  Viewer 
provides  ready  ‘drill-down’  capabilities  via  the  buttons 
arrayed  to  either  side  of  the  central  display.  This  allows 
planners  to  access  additional  information  (e.g.,  airfield 
restrictions,  clearance  requirements,  full  data  on  any 
mission  selected)  without  having  to  call  up  another 
interface  unit  to  execute  additional  queries  against  one  or 
more  databases. 
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The  data  necessary  to  achieve  this  concise  overview  is 
currently  distributed  in  numerous  record  fields  among 
multiple  databases.  Some  of  the  data  required  to  ‘draw  a 
picture’  for  the  user  is  not  stored  in  accordance  with  the 
user’s  ‘semantics’  at  all,  and  must  be  interpreted  through 


The  Smart  Lieutenant  palette  provides  the  mission 
planner  with  a  single  display  from  which  he/she  can 
access  all  other  relevant  classes  of  display  elements. 
Records  of  missions  (either  specific  ones  or  all  missions 
for  this  planner)  can  be  invoked.  Alerts  generated  by 
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Figure  2:  The  Port  Viewer 


inference.  The  invocation  of  ASL  layer  agents  allows  for 
this  necessary  data  access,  fusion,  and  interpretation  out 
of  sight  of  the  end  user.  This  avoids  confusing  the  user 
with  unnecessary  details  in  the  data  stream  itself,  as  well 
as  relieving  the  user  of  cognitive  burdens  associated  with 
manipulating  the  mechanics  of  the  database  and/or 
making  sense  of  the  data  received. 

To  summarize,  the  Port  Viewer  provides  a  unified,  fused 
data  display  configured  to  reflect  the  user’s  task 
ontology,  absent  superfluous  details  and  interpretational 
cognitive  burdens  which  increase  the  potential  for  errors. 
It  affords  the  user  referential  homogeneity  (simplicity) 
with  respect  to  data  sources  of  high  heterogeneity.  It 
accomplishes  this  by  according  agents  autonomy  to 
perform  the  requisite  data  retrieval  and  fusion.  In 
addition  to  satisfying  the  design  criteria  listed  above  for 
the  general  task  ontology  development,  the  Port  Viewer 
illustrates  minimum  procedural  costs  for  accessing  and 
retrieving  relevant  data  as  well  as  maximum  effective 
fusion  of  data  from  multiple  sources. 

The  Port  Viewer  concept  has  received  positive  feedback 
and  acceptance  from  the  planning  personnel  to  whom  it 
has  been  demonstrated.  The  key  to  this  ‘payoff  has  been 
our  ability  to  offer  HCI  layer  elements  consistent  with  the 
ontology  of  the  user’s  work  and  not  constrained  by  the 
ontology  of  the  supporting  system(s). 

IV. C..  Homogeneity  of  User  Work  Engagement: 
The  Smart  Lieutenant  Palette 

The  most  striking  characteristic  of  channel  mission 
planners’  information  systems  support  was  its  extreme 
heterogeneity.  There  was  no  single  ‘entry  point’  into  the 
complexities  of  the  mission  planning,  problem 
identification,  replanning,  and  mission  execution  tasks. 
Our  HISA  interface  architecture  provided  such  an 
integrated  entry  point  via  the  ‘Smart  Lieutenant’  palette 
illustrated  in  Figure  3  below. 


intelligent  ASL  agents  can  be  managed  through 
invocation  of  a  pending  alert  queue  or  a  historical  listing 
of  past  alert  conditions.  Indicators  on  the  palette  cue  the 
planner  to  the  presence  of  pending  alerts,  as  well  as  the 
arrival  of  new  alerts  since  he/she  last  reviewed  the  alert 
queue.  In  a  similar  fashion,  the  planner  is  allowed  to 
manage  the  stream  of  incoming  queries  and  reports 
(automated  stock  queries),  as  well  as  to  invoke  a  Query 
Assistant  to  generate  new  queries.  Finally,  a  set  of  tool 
options  allow  the  planner  to  inspect  and/or  manipulate 
agents,  contacts,  and  preferences. 
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Figure  3:  The  Smart  Lieutenant  Palette 
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The  Smart  Lieutenant  palette  enforces  referential  and 
procedural  homogeneity  in  the  user’s  engagement  with 
the  obvious  heterogeneity  of  his/her  support  agents  and 
the  mission  planning  information  systems.  As  the  top- 
level  procedural  portal,  this  interface  element  is  the  one 
most  reflective  of  system  semantics  (e.g.,  queries, 
agents).  However,  this  invocation  of  system  ontology  is 
The  Smart  Lieutenant  palette  reflects  our  design  criterion 
of  maximum  cross-reference  among  HISA  elements  with 
respect  to  the  task  domain  constructs  of  (e.g.)  information 
requests,  pending  issues,  and  workload  management 
parameters.  It  maximizes  effective  fusion  of  procedural 
data  into  one  concise  portal  for  subsequent  drill-down.  It 
minimizes  procedural  costs  by  affording  direct  drill-down 
to  key  task  elements.  It  minimizes  cognitive  complexity 
by  summarizing  the  existence  of  alerts,  queries,  reports, 
and  relevant  agents. 

V.  Conclusion 

We  began  this  paper  by  discussing  the  command  and 
control  system  of  the  future — the  JBI — and  some  of  the 
challenges  and  opportunities  it  affords  HCI  designers. 
We  then  discussed  the  role  that  interface  agents  will  play 
in  creating  an  environment  that  enables  a  user  to  remain 
“on  task”  longer,  and  concretized  the  discussion  by 
providing  example  interfaces  from  the  HISA  effort.  In 
this  final  section,  we  discuss  the  challenges  of  developing 
direct  manipulation,  work  centered,  interfaces  for  a  full 
vision  JBI.  Up  to  this  point,  we  have  characterized  the 
JBI  mainly  in  terms  of  three  different  services  layers.  Our 
discussion  has  concentrated  on  an  agent-based  direct 
manipulation  interface  concept  in  the  context  of  a 
distributed  network-centric  architecture  focused  on  airlift 
command  and  control.  It  is  important  to  recognize, 
however,  that  the  full  vision  of  a  JBI  involves  a  diverse 
collection  of  network  centric  systems  that  integrate  air 
and  space  operations.  One  should  ask  if  the  agent-based 
work  centered  interface  concept  can  scale  to  meet 
information  usability  needs  for  a  full-blown  JBI. 

The  JBI  Information  Technology  concept  consists  of  a 
core  network  system  designed  from  the  perspective  of 
supporting  battle  management  activities  within  an  Air 
Operations  Center  framework.  The  principal  goal  is  to 
enable  the  Joint  Force  Component  Commander  and 
supporting  staff  to  make  well-informed  decisions  that  can 
be  executed  rapidly  in  a  highly  coordinated  manner. 
Space  operations,  airlift,  logistics,  intelligence,  and 
network  security  are  all  elements  that  support  contingency 
operations.  Each  of  these  areas  of  the  military 
organization  are  represented  both  in  the  core  JBI  system 
and  via  links  to  the  extensive  information  networks 
maintain  in  each  separate  area.  The  essential  idea  for  the 
JBI  is  that  a  core  information/command  and  control 
system  will  be  operational  to  support  the  commander 
within  hours  after  approval  for  a  contingency  operation. 
Multiple  JBIs  may  be  in  commission  and  operated 


itself  constrained  with  respect  to  task-specific  features. 
The  alert  queue  provides  a  4to-do  list’  by  which  the 
planner  can  organize  his/her  daily  itinerary.  Similarly, 
the  query  features  are  offered  with  respect  to  the 
planner’s  work  flow  and  activity  history,  and  agents  can 
be  called  up  based  upon  their  participation  in  a  specific 
task  event  (mission  display,  alert  notification), 
simultaneously,  each  sharing  a  common  web  with  links  to 
information  and  other  resources  provided  by  the  same 
support  agencies.  In  some  sense,  each  JBI  will  be  a 
“virtual”  organization  pulling  on  assets  from  every 
available  source  regardless  of  its  physical  location. 

Clearly  the  level  of  information  management  associated 
with  the  JBI  concept  is  unprecedented  for  military 
operations.  Rather  than  reducing  the  “fog  of  war”  it 
could  in  fact  equally  as  well  contribute  to  it.  It  order  to 
insure  that  the  JBI  achieves  its  goal  as  a  work  support 
system,  we  believe  the  user  interfaces  each  member  of  the 
JBI  staff  must  also  be  regarded  as  a  support  system  that  is 
organized  in  a  manner  that  keeps  the  worker  maximally 
“on-task”  even  as  the  characteristics  of  the  work  problem 
changes  based  on  prevailing  conditions.  The  agent-based 
direct  manipulation  interface  attempts  to  achieve  this  goal 
by  insuring  the  visible  portion  of  the  interface  follows  a 
stable  and  consistent,  yet  flexible,  work-oriented  ontology 
that  can  dynamically  connect  to  any  appropriate 
information  source  through  an  interface  agent  that 
mediates  ontologically  differences  with  delivery  agents. 
The  homogeneous  work  centric  interface  focus  is 
maintained  even  as  the  user  finds  the  need  to  drill  down 
for  more  detail  or  drill  in  to  inspect  and  evaluate  vital 
aspects  of  information  sources,  which  results  in  dynamic 
connections  to  a  pool  of  heterogeneous  server  agents, 
data  sources,  and  application  tools. 

It  should  be  clear  that  on  conceptual  grounds  our 
interface  concept  scales  to  the  larger  arena  of  full 
battlespace  management.  However,  on  a  practical  levels 
the  design  task  may  be  more  challenging.  One  issue 
revolves  around  the  semantic  mapping  from  an 
information/application  tool  domain  to  the  work  centric 
one  of  the  user.  The  range  of  information  types  and  tools 
will  become  larger  and  more  diverse.  Can  effective 
semantic  maps  be  found  for  all  of  them?  Clearly  it  would 
be  desirable  if  we  could  establish  and  validate  semantic 
mapping  principles  that  could  be  used  to  accomplish  this 
task.  A  related  issue  deals  with  the  extent  of  automation 
present  in  the  software  interface  mediators.  In  order  to 
achieve  the  desired  semantic  mapping,  interface  agents 
may  have  to  take  on  more  functions  that  will  be  opaque  to 
the  user.  This  increases  the  likelihood  of 
miscommunication  of  the  interface  to  the  worker— the 
problem  of  automation  surprise  (Woods,  Sarter,  and 
Billings,  1997).  Can  this  problem  be  avoided?  More 
research  may  have  to  be  directed  in  this  area. 
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To  date,  we  have  completed  a  preliminary  demonstration 
of  our  agent-based  direct  manipulation  concept.  Initial 
reaction  has  been  very  favorable,  and  a  second 
demonstration  is  scheduled.  However,  to  more 
thoroughly  evaluate  the  concept,  an  experiment  is  needed 
that  as  a  minimum  measures  the  predicted  on-task/off-task 
time  advantage  and  correlates  it  with  a  mission 
performance  metric.  Further,  additional  research  will  be 
needed  to  address  the  implications  for  maintaining  a  work 
centric  interface  focus  as  the  properties  of  the  interface 
itself  expand  to  include  such  things  as  multi-media,  multi¬ 
modal,  and  adaptive  characteristics.  Can  these  properties 
be  enfolded  into  the  agent-based  direct  manipulation 
concept?  What  impact  might  they  have  on  semantic 
mapping? 

Our  agent-based  direct  manipulation  interface  is  the  first 
attempt  we  are  aware  of  to  propose  a  concept  for  how  to 
design  a  collected  set  of  work  centric  interfaces  to  a 
heterogeneous  information  network.  It  goes  beyond  the 
issue  of  standard  “look  and  feel”  that  dominates  user 
interface  design  today.  While  it  may  not  be  the  final 
answer,  we  believe  it  is  at  least  a  useful  first  step  to 
enable  the  JBI  vision  from  the  perspective  of  each 
individual  user 
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Summary:  Military  systems  at  all  levels  of 
decision  making  require  the  ability  for  the  decision 
maker  to  find  information,  query  it,  seek  refinement 
of  it,  process  it  in  combination  with  other 
information,  add  value  to  it,  make  the  decision  and 
communicate  the  decision  to  another  user.  The  US 
Office  of  Naval  Research  program  in  Interactive 
Multimedia  and  User-Centered  Design  supports 
research  in  how  to  employ  technological 
capabilities  to  enhance  a  person’s  abilities  to  carry 
out  the  decision  making  objective.  This  paper  will 
employ  several  research  projects  from  this  program 
to  illustrate  basic  findings  that  impact  how  to 
design  systems  to  meet  this  objective.  Usability 
objectives  require  that  design  address  the  impact  on 
the  user’s  ability  to  perform  the  task.  The  research 
reported  here,  even  though  in  domains  or 
applications  that  differ  from  battle  management  are 
at  the  level  of  studies  of  enabling  understanding  of 
the  design  space.  Reported  results  provide 
guidance  and  suggest  how  to  design  the  information 
presentation  in  the  appropriate  form  for  its  use. 

First,  basic  assumptions  behind  development  of  the 
program  will  be  described  to  set  the  context  for  the 
projects.  Then,  each  project  will  be  summarized 
presenting  the  research  findings.  The  selected 
projects  provide  scientific  bases  for  design 
decisions  that  impact  how  a  person  can  actually  use 
information  in  different  presentation  contexts, 
multimedia  documents,  multidimensional  flat  panel 
displays  and  in  a  Responsive  Workbench  context. 
Other  research  includes  multiple  modalities  but  will 
not  be  discussed  here. 

Background:  The  program  in  Interactive 

Multimedia  and  User-Centered  Design  assumes  an 
integrated  multidisciplinary  approach  to  employing 
cognitive  modeling  and  perceptual  understanding  in 
design  of  task- focused  interactive  systems.  Teams 
of  researchers  include  cognitive  scientists. 


psychologists,  computer  scientists,  and  experts  in 
experimental  design  in  differing  combinations. 
Research  projects  are  theory  and  hypothesis  based 
allowing  effective  evaluation  to  produce  better 
understanding  of  the  human  component  of  a  system 
and  hence,  contributing  to  effective  system 
usability  and  design.  Computational  cognitive 
models  such  as  ACT-R  (Anderson,  1993  )  , 
COGNET  (  Zachery,  Ryder  and  Hicinbothom, 
1999)  and  EPIC/GLEAN  (Meyer  and  Kieras, 
1997a,  1997b)  are  being  developed  to  provide 
resources  for  cognitive  appraisal  of  design 
decisions. 

In  addition,  building  usable  systems  entails  analysis 
of  which  modalities  are  best  for  what  types  of 
information  or  information  use,  such  as  vision, 
audition,  or  somatosensory,  and  what  media  can 
provide  them,  such  as  a  monitor,  film,  3-D  sound, 
etc.  The  space  is  further  complicated  in  that 
information  presentation  can  have  multiple  modes 
that  also  differ  in  how  a  human  can  deal  with  them, 
such  as  in  the  visual  modality  on  a  monitor,  one  can 
have  graphs  and  text,  or  text  and  an  animated 
drawing  or  only  text  as  modes.  These  three 
dimensions,  modality,  media  and  mode  are  being 
investigated  for  better  understanding  of  their 
impact  on  the  user’s  ability  to  deal  with  the 
information  being  presented. 

With  this  brief  introduction  of  some  of  the  features 
of  the  design  space  that  must  be  addressed  to  obtain 
effective  design,  the  remainder  of  the  paper  will 
illustrate  how  some  of  them  effect  usability  of 
information  by  presenting  research  investigating: 
Interactive  multimedia  document  design;  Visual 
presentation  of  information  using  perspective  view 
technology,  2-D  or  3-D  on  flat  panels;  and 
Usability  in  Virtual  Environments  (VE),  user- 
focused  information  management  in  an  immersive 
Responsive  Workbench  environment. 


Paper  presented  at  the  RTO  HFM  Symposium  on  “Usability  of  Information  in  Battle  Management 
Operations”,  held  in  Oslo ,  Norway,  10-13  April  2000,  and  published  in  RTO  MP-57. 
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Interactive  Multimedia  Document  Design:  Dr. 

Mary  Hegarty  (University  of  California  Santa 
Barbara)  and  Dr.  N.  Hari  Narayanan  (Auburn 
University)  have  teamed  to  investigate  a  cognitive 
theoretical  model-based  multimedia  design  for 
manuals  that  are  intended  to  teach  how  devices 
work.  Such  manuals  are  a  focus  of  many  new 
requirements  that  insist  on  on-line  information  for 
training  and  reference.  They  are  developing 
methods  for  presenting  information  to  maximize 
the  user’s  ability  to  understand  causal  relationships 
from  it.  Their  theory  includes  a  model  of  human 
states  of  comprehension  of  mechanical  devices  (the 
domain  of  previous  work)  and  their  operations. 
The  theory  includes  a  four  step  process  in  obtaining 
understanding  of  how  a  device  works  to  be  able  to 
apply  the  understanding  to  a  real  world  instance  of 
the  device.  The  theory  of  comprehension  includes 
the  following  stages:  Stage  1:  Machine 

Decomposition  by  diagram  parsing;  Stage  2: 
Constructing  a  Mental  Model-  Making 
Representational  Connections;  Stage  3: 
Constructing  a  Static  Mental  Model-  Making 
Referential  Connections;  Stage  4:  Determining  the 
Causal  Chain  of  Events;  and  Stage  5: 
Constructing  a  Dynamic  Mental  Model  by 
Mental  Simulation  and  Rule-based  Inference 
(Narayanan  and  Hegarty,  1998). 

Their  method  for  design  recognizes  cognitive 
workload  factors  and  attempts  to  reduce  them 
during  reading  and  studying  of  the  material.  They 
are  studying  how  the  integrated  placement  of  text 
explanations  and  diagrams,  both  static  and 
animated,  contributes  to  the  understanding  of 
concepts  and  processes  in  a  hyperlinked  document. 
The  model  is  based  on  previous  work  on  relative 
location  of  text  and  graphics,  showing  that  text 
associated  with  an  image  must  be  nearby  and  that 
labeling  in  the  text  must  be  clearly  visible  on  the 
drawing  (Hegarty,  1992;  Mayer,  1989).  They 
designed  on-line  documents,  based  on  the  theory  of 
stages  just  described,  in  the  context  of 
understanding  how  a  flushing  cistern  (toilet)  works. 
Understanding  requires  inferring  causal 
relationships  after  the  components  are  recognized 
and  integrating  them  with  world  knowledge  about 
water  flow  and  how  the  mechanical  components 
work  in  isolation. 

The  initial  design  of  the  hypermedia  manual 
integrating  this  theory  produced  a  complex  set  of 


experiments  to  validate  the  theory.  The  studies 
compared  learning  and  understanding  using  a 
hyperlinked  version  of  the  information  as  well  as  a 
static  paper  version,  both  were  created  using  the 
theory  of  comprehension  summarized  above.  The 
results  indicate  that  presenting  information  in  a 
hypermedia  manual  that  includes  hyperlinks, 
colored  diagrams,  animations  (rather  than  static 
diagrams)  and  commentaries  (rather  than  visual 
text)  had  no  effects  on  comprehension.  The  same 
results  were  obtained  by  expository  causal  link 
information  in  the  text  and  in  some  cases  just  a 
labeled  diagram  alone  conveyed  the  same  level  of 
understanding.  It  seems  that  the  juxtaposition  of 
the  structure  and  content  of  closely  associated 
material  is  more  important  than  the  form  of  the 
document(  Hegarty,  Quilici,  Narayanan,  Holmquist 
and  Moreno,  1999). 

Among  their  findings  is  the  fact  that  they  were 
unable  to  adequately  evaluate  their  theory  and 
model  of  design  and  its  predictions  in  the  context  of 
the  document.  Analysis  of  their  results  have 
suggested  other  ways  to  implement  their  principled 
theory  in  the  context  of  interactive  multimedia 
information  presentations.  Current  work  continues 
to  find  that  the  “message  is  more  important  than  the 
medium”  as  far  as  using  technology  to  enhance 
understanding  or  comprehension. 

Additionally,  Hegarty  and  Narayanan  found,  as  has 
been  documented  elsewhere,  that  animation  does 
not  necessarily  aid  understanding  and  that  it  can 
confuse  the  user.  Several  results  indicate  that 
animation  must  be  well  integrated  with  text,  or 
needs  to  be  integrated  into  material  in  a  manner  that 
requires  interaction  with  the  concepts  in  addition  to 
exploration  of  the  animated  rendering  of  it  (Hegarty 
and  Sims,  1994;  Kehoe,  Stasko,  and  Taylor,  1999). 

They  are  continuing  their  investigation  on 
understanding  multimedia  information  using 
atmospheric  weather  as  their  focus.  It  is  something, 
not  directly  observable  but  often  visualized  for 
understanding,  such  as  weather  front  movements 
and  wind  speeds  and  directions.  Users  will  be 
evaluated  for  their  retention  of  information  and 
understanding  of  causal  relationships  somewhat 
similar  to  the  original  study  of  how  a  pumping 
cistern  works. 

Others  have  been  studying  the  effectiveness  of 
animation  in  learning  contexts.  The  fact  that  its 
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role  is  controversial  even  though  assumed  to 
enhance  learning  has  implications  for  the  use  of 
animation  in  battle  management  contexts.  Dr.  John 
Stasko  and  his  Information  Visualization  Group 
(Georgia  Institute  of  Technology)  has  shown  that 
animation  improves  understanding  in  certain 
contexts  and  has  been  exploring  how  to  best 
provide  animation  capabilities  in  learning  systems 
(Stasko,  1996). 

Empirical  evidence  suggests  that  application  of 
animation  in  learning  and  possibly  in  understanding 
information  in  decision  making  environments  may 
depend  on  knowledge  of  the  context  of  application, 
the  mental  workload  of  the  person,  and  whether  the 
person  has  the  ability  to  mentally  visualize  and 
dynamically  manipulate  it.  In  understanding 
algorithms,  or  software  processes,  when  the  person 
already  understands  a  principle  and  is  able  to 
somehow  visualize  it,  the  animation  seems  to 
provide  enhanced  learning  (Kehoe,  Stasko,  and 
Taylor,  1999).  The  animation  provides  additional 
depth  of  understanding.  This  was  demonstrated  in 
the  context  of  animations  to  explain  how  software 
algorithms  worked,  a  phenomena  that  has  no  real 
world  visualization.  It  seems  that  animation  is  only 
useful  as  an  enhancement  to  understanding  when 


the  person  is  already  able  to  mentally  visualize  the 
action  or  concept. 

Visual  Presentation  of  Information  Using 
Perspective  View  Technology:  Drs.  Michael 
Cowen  (SPAWAR  Systems  Center-San  Diego)  and 
Mark  St.  John  (Pacific  Science  and  Engineering 
Group,  Inc.)  are  investigating  the  role  display 
formats  play  in  the  ability  of  a  user  to  access  and 
use  information.  The  issue  can  be  stated  as,  what 
type  of  information  is  best  presented  in  what 
format?  In  a  recent  technical  report  (St.  John  and 
Cowen,  1999)  results  from  experiments  comparing 
two  presentation  formats  to  determine  answers  to 
the  question  about  form  and  available  content  have 
shown  that  in  tasks  where  relative  position  is 
important,  the  multiple  projection  2-D  presentation 
provides  explicit  information  that  can  easily  be 
determined  with  little  error.  In  contrast,  the  3-D 
presentation  has  increased  ambiguity  due  to  the 
perspective  rendering  of  the  same  information.  See 
Figure  1.  If  the  task  is  to  determine  the  shape  or 
general  configuration  of  an  object,  then  the  3-D  is 
superior  to  the  2-D  multiple  view  perspective  (St. 
John  and  Cowen,  1999). 
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Based  on  these  results,  subsequent  investigations 
were  carried  out  to  determine  the  implications  of 
these  findings  in  an  Area  Air  Defense  Display.  The 
system  design  included  several  3-D  presentations, 
of  terrain  and  of  icons,  and  the  goal  was  to 


determine  whether,  the  3-D  perspectives  and 
realism  enhanced  situation  awareness  or  merely 
provided  a  more  satisfying  feel  to  the  picture. 
Examples  of  the  icons  used  in  the  experiment  are 
provided  in  Figure  2. 


Figure  2:  Examples  of  Icon  conditions  for  experiment,  left  to  right,  East  heading  bomber  depicted 
realistically  in  3-D,  realistically  in  2-D,  and  symbolically  or  non-realistically  (Smallman,  Schiller,  and 

Mitchell,  1999). 


The  icons  were  overlaid  on  a  flat  panel  3-D  terrain 
map  in  perspective  view  in  one  condition,  and  in 
planar  view  in  the  other  two  conditions.  Tasks 
include  track  identification  and  monitoring  to  gain 
situation  awareness.  Relative  ground  position  was 
provided  by  the  corresponding  shadow  on  the 
terrain  surface  in  the  perspective  view  condition. 
The  scenario  used  was  the  same  for  all  three 


conditions,  1)  3-D  realistic  icons  on  a  3-D 
perspective  terrain  presentation,  2)  3-D  realistic 
icons  on  a  planar  terrain  presentation  ( no  shadows 
on  surface  and  bird’s  eye  view)  and  3)  2-D  non- 
realistic  icons  on  a  planar  terrain  presentation. 
Figure  3  shows  the  2-D  iconic  representation  on  a 
planar  terrain  presentation  as  an  example,  condition 
3  just  discussed. 


Figure  3:  2-D  non-realistic  icon  view  on  a  planar  view  map  (Smallman,  Schiller,  and  Mitchell,  1999). 


Results  showed  that  the  2-D  iconic  representations 
are  the  easiest  to  remember  and  produce  the  fewest 
errors  on  identification  and  reporting  of  the  tested 
attributes.  While  the  3-D  icon  on  the  3-D 


perspective  terrain  presentation  was  assumed  by  the 
design  to  provide  the  best  depiction  to  facilitate  the 
user  task,  the  results  showed  that  the  opposite 
holds.  Due  to  the  foreshortening  of  the 
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perspectives  and  the  nature  of  the  shadow 
connectivity  with  the  icon,  ambiguities  are 
introduced  that  produce  errors  in  the  ability  of  the 
user  to  track  the  attributes  of  the  object  represented 
by  the  icon.  In  addition,  when  the  icons  are  small 
due  to  the  distance  above  the  ground,  it  is  very 
difficult  to  find  the  corresponding  shadow  for  the 
aircraft.  Decluttering  the  image  may  provide  some 
help,  but  adds  another  subtask  for  the  user. 
Furthermore,  in  the  3-D  case,  the  light  source  and 
its  effect  on  how  the  shadows  appear  introduces 
another  aspect  of  ambiguity  in  the  presentation. 
Details  of  these  investigations  and  full  results  can 
be  found  in  Smallman,  Schiller,  Mitchell,  1999. 

Battle  management  systems  require  many  tasks  and 
manipulations  of  information  similar  to  those  that 
were  used  in  these  studies.  The  usability  issues 
scientifically  documented  in  these  studies  are 
important  and  generally  ignored  in  current  designs. 
Understanding  limitations  on  visual  presentation 
options  and  how  they  interact  with  user  tasks  and 
with  user  capabilities  needs  to  be  further  explored. 

Usability  in  Virtual  Environments:  On-going 
joint  research  by  Dr.  Deborah  Hix  at  Virginia 
Polytechnic  Institute  and  State  University  ( 
Virginia  Tech)  and  by  Dr.  Edward  Swan  II  at  the 
Naval  Research  Laboratory  (NRL)  focuses  on 
methods  for  defining  usability  criteria  in  virtual 
environment  (VE)  design.  Their  recent  award 
winning  paper  at  the  Virtual  Reality  99 
Conference,  (Hix,  Swan,  Gabbard,  McGee,  Durbin 
and  King,  1 999)  exemplifies  a  four  step  method  of 
usability  design  that  was  verified  while  applying  it 
in  a  Naval  Battlespace  context.  This  section  will 
begin  with  an  overview  of  a  preliminary 
investigation  into  the  state  of  usability  for  VE  and 
will  conclude  with  an  overview  of  the  methodology 
and  its  application  in  the  battle  management 
domain. 

Initial  questions  regarding  usability  issues  in  VE 
arose  during  discussions  about  device  selection  and 
use  and  the  fact  that  the  metaphors  for  use  in  VEs 
are  far  more  complex  and  allow  many  new 
interaction  possibilities  than  any  other  technology 
to  date.  Additional  evidence  that  human  usability 
issues  need  attention  has  been  documented  in  the 
findings  that  humans  subjected  to  VEs  suffer 
performance  effects  even  up  to  an  hour  after  they 


leave  the  immersion  (Kennedy  and  Stanney,  1996). 
Previous  enhancements  to  VE  technology  that 
focused  on  presentations,  visual,  auditory,  haptic, 
and  their  refinement  was  able  to  only  ameliorate 
part  of  the  effect  on  the  user.  Since  the  VE  provides 
an  immersive  interactive  space  for  design  and  use, 
it  presents  a  design  challenge  along  a  vast  number 
of  dimensions. 

An  initial  effort  to  document  a  taxonomy  of 
usability  characteristics  for  VEs  intended  to  begin  a 
dialogue  that  would  aid  in  understanding  the  design 
space  at  a  theoretical  level  and  that  would  aid  in 
effective  VE  design.  Dr.  Hix  with  her  then 
graduate  student,  Joseph  Gabbard,  began  their 
effort  to  understand  the  usability  issues  regarding 
design  of  VEs.  They  did  a  thorough  review  of 
relevant  literature,  interviewed  leading  VE 
researchers  and  designers,  and  talked  with 
practitioners  and  users  of  VE  systems,  chiefly  in 
training  or  mission  rehearsal  domains.  They  found 
that  the  usability  space  for  VEs  included  complex 
interdependencies,  among  users,  user  tasks,  input 
devices,  output  devices,  etc.  The  result  of  their 
investigation  is  a  taxonomy  of  usability  methods 
(Gabbard  and  Hix,  1 997). 

Employing  known  usability  methods  as  a  start 
point,  they  found  that  VE  expanded  their 
applicability  to  teams  and  integrated  team  tasks,  as 
well  as  tasks  that  involved  the  whole  human  as 
opposed  to  using  a  system  while  sitting  in  front  of  a 
monitor.  Immersion  as  part  of  usability  requires 
new  metaphors  in  thinking  about  what  usability 
means.  The  framework  in  the  document  organizes 
user  interaction  design  guidelines  and  discussion 
into  four  major  areas:  1)  users  and  user  tasks,  2) 
input  mechanisms,  3)  virtual  models,  and  4) 
presentation  mechanisms.  All  findings  are  cross- 
linked  and  documented  with  interview  explanations 
or  cross-referenced  to  other  related  usability.  The 
document  intended  as  a  dynamic  taxonomy, 
provides  a  broad  definition  of  the  state  of  usability 
in  VEs.  It  is  available  as  a  public  resource  at  URL: 

http://csgrad.cs.vt.edu/~igabbard/ve/taxonomv.  Figure 
4  provides  an  overview  of  the  taxonomy  areas. 
This  characterizes  the  complex  design  space  and 
illustrates  how  usability  can  impact  the  design  of 
VEs. 
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Figure  4:  An  Overview  of  the  Taxonomy  (Gabbard  and  Hix,  1997). 


Follow-on  joint  work  at  NRL  integrated  many  of 
the  basic  findings  into  a  defined  method  for 
usability  evaluation  in  the  context  of  battle 
information  management.  This  work  employed  an 
immersive  environment  on  an  Responsive 
Workbench.  In  addition  to  developing  a  suitable 
interaction  device,  the  effort  took  approximately 
nine  months  to  apply  the  usability  method, 
validating  it  as  the  work  progressed  and  in  the  end, 
producing  a  system  that  is  ready  for  summative 
evaluation  in  the  application  domain. 

This  development  has  supported  the  fact  that  by 
carrying  out  the  up-front  usability  design  methods, 
life-cycle  cost  reduction  can  be  realized.  A  few  of 
the  many  discoveries  in  the  context  of  this  work 
will  be  summarized.  Below,  Figure  5,  is  the 
workbench  environment,  Dragon  (Durbin  et  ah, 
1998),  including  the  terrain  and  object  renderings 
that  form  the  basis  of  the  design  solution.  It  is  the 


result  of  a  successful  application  of  the  design,  test, 
design  cycle  of  usability  assessment.  Details  of  the 
effort  can  be  found  in  the  Virtual  Reality  99  paper 
referenced  above.  The  methodology  is  elaborated 
in  Gabbard,  Hix  and  Swan,  1999.  The  usability 
method  includes  a  four  step  assessment,  based  on 
the  design  space  presented  in  Figure  4.  The  user- 
centered  methodology  assumes  sequential 
performance  of  the  following  4  steps:  1)  user  task 
analysis,  2)  expert  guidelines-based  evaluation,  3) 
formative  user-centered  evaluation,  and  4) 
summative  evaluation. 

Cyclic  evaluations  are  performed  as  design 
problems  arise.  As  more  problems  are  eliminated 
in  the  early  phases,  the  cost  of  implementation  and 
subsequent  possible  modification  are  reduced.  The 
four  steps  will  be  briefly  described  In  the  context  of 
developing  the  interface  design  for  navigating 
within  the  Dragon  workbench  environment. 
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Figure  5:  Dragon  Responsive  Workbench  Design  (Hix  et  al.,  1999) 


1. 

1.  User  task  analysis:  must  include  an 

understanding  that  is  as  comprehensive  as 
possible  of  the  tasks  the  user  will  be  doing  and 
the  goals  of  the  total  system  use  to  achieve 
good  user-centered  design.  It  was  found  very 
early  in  working  with  the  workbench,  that  the 
ability  of  the  user  to  move  around,  locomote,  to 
find  the  information  they  need  is  critical  to  any 
other  tasks.  Therefore,  the  task  selected  to 
implement  and  evaluate  the  method  became 
locomotion. 

2.  Expert  guidelines-based  evaluation: 

identifies  potential  usability  problems  by 
evaluating  a  user  interaction  design  against 
what  is  known  by  experts  to  be  sound  design 
principles.  Any  violations  are  revisited  and  the 
design  is  redone  to  address  them.  Multiple 
experts  perform  the  evaluation,  independently 
and  then  in  joint  discussion.  During  this  phase 
in  the  design  of  Dragon,  experts  discovered  a 
major  problem  of  poor  mapping  of  locomotion 


tasks  (pan,  zoom,  pitch,  heading)  to  flight  stick 
buttons  as  well  as  problems  with  inadequate 
graphical  and  textual  feedback  to  the  user  about 
the  locomotion. 

3.  Formative  user-centered  evaluation:  requires 
careful  definition  of  scenarios  that  will  exercise 
the  interface  under  the  set  of  tasks  included  in 
the  design.  In  the  Dragon  design  of 
locomotion,  the  initial  control  was  to  allow  the 
user  to  move  the  map  relative  to  themselves. 
During  formative  evaluation,  users  indicated 
they  wanted  to  be  able  to  fly  over  the  terrain. 
This  input  suggested  a  different  design  that  was 
developed  to  allow  movement  in  exocentric 
(the  map  movement)  and  egocentric  ways(  the 
fly  -over).  Subsequent  evaluations  were 
carried  out  to  build  the  right  responsiveness 
into  the  interface  to  adequately  capture  the 
control  from  the  user’s  perspective  for  both 
these  metaphors.  This  evaluation  includes  data 
collection  about  user’s  performance  that  guides 
the  design  for  the  system. 
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4.  Summative  evaluation:  requires  comparative 
evaluation  between  several  competing  designs. 
This  is  a  formal  evaluation  of  the  implemented 
user-focused  interface.  The  affects  of 
exocentric  vs  egocentric  control  are  one 
variation  being  studied  in  Dragon.  To 
effectively  test  this,  all  actions  must  be 
accomplishable  in  both  systems  to  be  able  to 
compare  measures  that  will  determine  the  best 
design.  Speed  of  problem  solving  is  one  such 
measure  as  is  correctness  of  solution.  For  this 
type  of  evaluation  an  adequate  scenario  test  set 
must  be  defined  that  drives  the  interface  use  in 
ways  that  will  stress  its  functionality  to  firmly 
determine  the  best  solution.  Summative 
evaluation  focusing  on  egocentric  versus 
exocentric  control  is  currently  underway 
(Gabbard,  Hix  and  Swan,  1999.) 

The  importance  of  this  applied  methodology  is  that 
the  right  level  of  detailed  evaluation  is  performed  at 
the  right  time  during  the  design,  saving  extensive 
redesign  to  correct  problems  late  in  the  cycle.  The 
possibility  of  achieving  an  adequate  and  effective 
system  is  greatly  enhanced  and  at  a  much  reduced 
overall  cost. 

Conclusions:  Usability  methods  have  many 

applications  in  military  systems.  We  are  just 
beginning  to  understand  some  of  the  underlying 
cognitive  aspects  that  are  reflected  in  how  people 
can  obtain  and  use  information.  As  more 
interdisciplinary  conversations  occur  between 
cognitive  scientists,  usability  engineers,  human- 
computer  interface  designers  and  domain  experts 
we  will  see  a  positive  change  in  the  design  of 
systems — more  usable  by  more  people  without 
restrictive  selection  criteria  and  the  overall  result 
will  be  a  cost  reduction  in  the  life-cycle  costs  of  the 
systems. 

References: 

Anderson,  J.  R.  (1993)  Rules  of  the  Mind, 
Erlbaum,  Hillsdale,  NJ. 

Durbin,  J.,  Swan  II,  E.,  Colbert  B.,  Crowe,  J.,  King, 
R,  King  ,T.,  Scannell,  C.,  Wartell,  Z.,  and  Welsh, 
T.  (1998)  “Battlefield  Visualization  on  the 
Responsive  Workbench,”  Proc.  IEEE  Visualization 
’98,  IEEE  CS  Press,  NC. 


Gabbard,  J.  L.  and  Hix,  D.  (1997)  A  Taxonomy  of 
Usability  Characteristics  in  Virtual 
Environments, 

On-line  report:  Virginia  Polytechnic  Institute  and 
State  University,  Blacksburg,  VA,  URL: 
http://csgrad.cs.vt.edu/--igabbard/ve/taxonomv/. 

Hegarty,  M.  (1992)  “Mental  animation:  Inferring 
motion  from  static  diagrams  of  mechanical 
systems,”  Journal  of  Experimental  Psychology: 
Learning,  Memory  and  Cognition,  18(5),  pp.  1084- 
1102. 

Hegarty,  M.,  Quilici,  J.,  Narayanan,  N.  H., 
Holmquist,  S.,  and  Moreno,  R.  (1999)  “Multimedia 
instruction:  Lessons  from  evaluation  of  a  theory 
based  design,”  Journal  of  Educational  Multimedia 
and  Hypermedia,  8(2)  pp.  119-150. 

Hegarty,  M.  and  Sims,  V.  K.  (1994)  “Individual 
differences  in  mental  animation  during  mechanical 
reasoning,”  Memory  and  Cognition,  22,  pp.  411- 
430. 

Hix,  D.,  Swan  II,  E.,  Gabbard,  J.  L.,  McGee,  M., 
Durbin,  J.  and  King,  T.  (1999)  “User-Centered 
Design  and  Evaluation  of  a  Real-Time  Battlefield 
Visualization  Virtual  Environment,”  Proc.  IEEE 
Virtual  Reality  99  Conf,  IEEE  CS  Press,  Los 
Alamitos,  CA,  pp.  96-103. 

Kehoe,  C.,  Stasko,  J.,  and  Taylor  ,A  (1999),” 
Rethinking  the  Evaluation  of  Algorithm 
Animations  as  Learning  Aids:  An  Observational 
Study,”  Technical  Report  GIT-GVU-99- 1 0, 
Georgia  Institute  of  Technology,  Atlanta,  GA. 

Kennedy  R.  S.  and  Stanney  K.  M.  (1996)  “Postural 
instability  induced  by  virtual  reality  exposure: 
Development  of  a  certification  protocol,” 
International  Journal  of  Human-Computer 
Interaction,  8(1),  pp.  25-47. 

Mayer,  R.  E.  (1989)  “Models  for  understanding,” 
Review  of  Educational  Research,  59,  pp.  43-64. 

Meyer  D.  E.  and  Kieras,  D.  E.  (1997a)  “A 
computational  theory  of  executive  control 
processes  and  human  multiple-task  performance: 
Part  1 .  Basic  Mechanisms,  “  Psychological  Review, 
104,  pp.  3-65. 


Meyer  D.  E.  and  Kieras,  D.  E.  (1997b)  “A 
computational  theory  of  executive  control 
processes  and  human  multiple-task  performance: 
Part  2.  Accounts  of  Psychological  Refractory- 
period  Phenomena,”  Psychological  Review,  104, 
pp.  749-791. 

Narayanan,  N.  H.  and  Hegarty,  M.  (1998)  “  On 
designing  comprehensible  interactive  hypermedia 
manuals,” 

International  Journal  of  Human-Computer  Studies, 
48,  pp.  267-301. 

St.  John,  M.  and  Cowen,  M.  B.  (1999)  “Use  of 
Perspective  View  Displays  for  Operational  Tasks,” 
Technical  Report  1795,  SPAWAR  Systems  Center 
San  Diego,  San  Diego,  CA. 

Smallman,  H.  S.,  Schiller,  E.  and  Mitchell,  C. 
(1999)  “Designing  a  Display  for  the  Area  Air 
Defense  Commander,”  Technical  Report  1803, 
SPAWAR  Systems  Center  San  Diego,  San  Diego, 
CA. 

Stasko,  J.  T.  (1996)  “  Using  Student-built 
Algorithm  Animations  as  Learning  Aids,” 
Technical  Report  GIT-GVU-96-19,  Georgia 
Institute  of  Technology,  Atlanta,  GA. 

Zachery,  W.  W.,  Ryder,  J.  M.,  and  Hicinbothom,  J. 
H.  (1998  )  “Cognitive  Task  Analysis  and  Modeling 
of  Decision  Making  in  complex  Environments,”  in 
Making  Decisions  Under  Stress,  Edited  by,  Janis 
A.  Cannon-Bowers  and  Eduardo  Salas,  American 
Psychological  Association,  Washington,  DC,  pp. 
315-344. 

URL  for  Chi  Systems  Inc.:  http://www.chiinc.com. 


21-1 


COMPONENTWARE  APPROACHES 
IN  MANAGEMENT  INFORMATON  SYSTEMS 

Dipl.-Ing.  Jurgen  Kaster,  Dipl.-Ing.  Annette  Kaster 

Ergonomics  and  Information  Systems  Department  (EFS) 

Research  Institute  for  Communication,  Information  Processing,  and  Ergonomics  (FKIE) 
Research  Establishment  for  Applied  Natural  Sciences  (FGAN) 

Neuenahrer  Str.  20,  D-53343  Wachtberg-Werthhoven,  Germany 
Tel:  [+49]  (228)  9435  -  380,  Fax:  -  508 
j.kaster@fgan.de,  a.kaster@fgan.de 


Summary 

Modern  command  and  control  information  systems 
(CCIS)  are  characterized  by  continuously  changing  con¬ 
ditions  regarding  technology,  task  and  user  profiles.  As 
a  consequence  of  this  heterogeneity  a  huge  amount  of 
information  and  knowledge  pieces  of  different  data  types 
has  to  be  managed  and  processed  in  distributed  commu¬ 
nication  networks.  The  situation  in  military  CCIS  is  even 
more  complex,  regarding  e.g.  new  requirements  and 
multi-national  command  structures  in  actual  out-of-area 
missions. 

The  paper  will  focus  on  architecture  models  on  the  basis 
of  " componentware  technology1 .  Pursuing  the  proposed 
ideas  may  help  to  design  systems  of  high  flexibility  that 
can  be  adapted  to  actual  user  needs  and  task  require¬ 
ments. 


1  Challenges  in  the  design  of  CCIS 

Military  CCIS  are  developed  to  solve  operational  re¬ 
quirements  in  the  areas  of  situation  identification  and 
assessment,  planning,  decision  making,  and  command¬ 
ing.  Appropriate  timely  information  flow  has  to  be  pro¬ 
vided  over  large,  distributed  communication  networks. 
However,  problems  in  actual  command  processes  are  not 
mainly  induced  by  the  complexity  of  military  structures. 
There  are  three  major  influence  coefficients  that  induce 
that  those  systems  are  unstable  over  time.  These  influ¬ 
ence  coefficients  are  rapidly  changing  conditions  re¬ 
garding  applied  technology,  operational  requirements, 
and  heterogeneous  user  profiles. 

In  most  cases  it  is  not  possible  to  design  military  infor¬ 
mation  systems  from  scratch  that  have  an  overall  con¬ 
sistent  hardware  and  software  basis  for  the  technical  and 
conceptual  implementation.  Operational  requirements 
cause  continuous  updates  and  adaptations.  As  a  conse¬ 
quence  a  rather  heterogeneous  environment  evolves  in 
which  command  and  control  processes  must  be  embed¬ 
ded.  Of  course,  this  complicates  the  realization  of  an 
universal  support  concept  and  induces  interruptions  of 
the  required  communication  flow.  Information  may  be 
falsified  or  even  lost. 


Another  problem  derives  from  the  fact  that  operational 
mission  requirements  have  dramatically  changed  in  the 
last  few  years.  Out-of-area  missions  have  to  be  planned, 
prepared  and  executed  within  a  short  time.  Task  forces 
have  to  be  established  in  accordance  to  actual  needs. 
This  requires  extremely  high  flexibility  and  adaptability 
of  the  underlying  technical  support  systems.  Addition¬ 
ally,  multi-national  missions  and  command  structures  are 
nowadays  a  matter  of  course. 

A  third  problem  area  is  the  need  of  supporting  total  dif¬ 
ferent  user  profiles.  Users  show  discrepancies  in  their 
technical  and  professional  competence,  working  styles, 
and  they  have  different  tasks  to  fulfill  at  the  same  system 
or  system  components. 

The  consequence  is  that  the  realization  of  universal  and 
stable  concepts  in  military  CCIS  is  very  difficult  or  even 
impossible.  The  well  known  problems,  which  are  para¬ 
phrased  by  "software  crisis",  are  established  mainly  by 
impediments  in  a  timely  and  task-oriented  access  to 
information  and  knowledge  that  is  stored  somewhere  in 
the  system. 

The  challenge  for  improvements  of  CCIS  from  a  human 
factors  point  of  view  is  to  provide  direct  problem- 
oriented  access  to  the  information  that  is  actually  needed 
in  current  operational  situations.  The  next  task  is  to  make 
the  human-machine  interface  as  homogeneous  as  possi¬ 
ble  even  in  modern  multimedia  environments.  The  rea¬ 
son  is  that  learning  processes  can  be  simplified  if  there  is 
a  high  value  of  recognition  implicitly  in  the  applied 
dialogue  procedures.  However,  one  has  to  take  into  ac¬ 
count  that  information  items  may  change  their  charac¬ 
teristics  during  processing  and  evaluation.  For  example, 
a  broad  range  of  textual  input  details  may  be  aggregated 
and  transformed  to  other  presentation  categories  to  sup¬ 
port  decision  processes  at  a  higher  command  level. 

To  overcome  restrictions  and  insufficiencies  in  military 
CCIS,  it  is  necessary  to  make  more  efficient  design  con¬ 
cepts  and  processes  available.  Componentware  techno¬ 
logy  is  a  new  paradigm,  that  allows  to  construct  flexible 
system  architectures  (macro-view)  as  well  as  concrete 
military  application  software  (micro-view).  The  follow¬ 
ing  chapter  will  focus  on  this  technology. 


Paper  presented  at  the  RTO  HFM  Symposium  on  “Usability  of  Information  in  Battle  Management 
Operations  ”,  held  in  Oslo ,  Norway ,  10-13  April  2000,  and  published  in  RTO  MP-57. 
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2  Componentware  -  a  new 

approach  for  CCIS  development 

2.1  Componentware  as  an  extension  of  object- 
oriented  techniques 

The  transfer  from  former  procedural  views  to  object- 
oriented  software  techniques  introduced  totally  new 
ways  of  thinking  and  abstractions  to  the  software  engi¬ 
neering  process.  Object-oriented  analysis  (OOA)  and 
object-oriented  programming  (OOP)  emphasize  concepts 
like  e.g.,  abstraction,  modularization,  and  reusability. 
However,  object-orientation  is  limited  to  the  reflection 
on  technical  structures  like  program  and  system  objects 
and  their  interrelationships. 

"As  a  system  becomes  more  complex,  the  design  prob¬ 
lem  goes  beyond  the  algorithms  and  data  structures  of 
the  computation:  designing  and  specifying  the  overall 
system  structures  emerges  as  a  new  kind  of  problem” 
[Booch  et.  al.  1999].  In  order  to  address  this  citation  it  is 
necessary  to  provide  methods  which  represent  a  system 
as  a  whole  assembled  by  independent  but  interacting 
components.  Frameworks  are  general  system  models 
which  structure  the  application  in  cooperating  building 
blocks.  Recently  the  componentware  principle  offers 
solutions  to  develop  applications  which  are  based  on 
such  frameworks  (Schreiber-Ehle  1999).  This  technique 
defines  standardized  protocols  which  independently 
developed  programs  can  use  to  communicate  (Microsoft 
1998;  OMG  1998).  If  this  technique  is  applied  modules 
can  be  exchanged  (even  at  runtime).  Therefore,  modifi¬ 
cations  of  the  system  and  its  performance  can  be 
achieved  even  after  the  software  installation  process  has 
been  completed  (Plasil  et.  al.  1998). 

A  software  component  is  an  executable  piece  of  software 
that  provides  a  useful,  in  the  application  context  valuable 
functionality.  It  offers  plug  &  play  readiness  for  service 
and  is  cooperative  in  combination  with  other  programs 
(Griffel  1998).  The  component  view  has  its  value  in  the 
emphasis  of  delimitations  and  independencies  of  soft¬ 
ware  elements  in  supplementation  to  internal  views  on 
program  and  system  objects.  Components  are  independ¬ 
ent  application-oriented  building  blocks  with  well  de¬ 
fined  interfaces  to  other  application  software.  Despite  the 
required  originality  of  a  software  component  it  becomes 
a  “real”  functional  component  (as  a  part  of  the  whole) 
only  in  combination  with  others.  This  will  result  in  a 
growing  significance  of  modern  framework  techniques 
(as  a  model  of  the  whole). 

The  component  view  may  be  applied  to  an  entire  infor¬ 
mation  system  as  well  as  to  special  application  software. 
This  dualism  is  also  important  considering  the  design 
pattern  which  is  introduced  in  the  next  chapter. 

2.2  Model-View-ControUer:  Design  pattern  for 
improved  data  management 

In  order  to  create  a  suitable  software  architecture  con¬ 
sisting  of  independent  exchangeable  parts  the  use  of 


design  patterns  is  helpful  (Gamma  et.  al.  1995).  Design 
patterns  structure  logical  dependencies  on  a  high  ab¬ 
straction  level.  They  are  applicable  to  a  software  design 
in  general  and  they  supply  designers  with  ready-made 
design  solutions.  A  useful  architectural  pattern  to  create 
componentware  systems  is  the  Model- View-Controller 
(MVC)  paradigm. 

An  optimization  of  user  interfaces  as  well  as  assistance 
systems  according  to  ergonomic  criteria  is  indispensable 
for  an  efficient  support  of  complex  command  and  control 
tasks  with  computer-based  information  systems.  There 
have  been  benefits  provided  by  the  construction  of  sepa¬ 
rate  user  interface  management  systems  (UIMS)  in  com¬ 
plex  applications  (Myers  1995).  However,  most  of  the 
applied  software  products  are  rigid  monolithic  blocks 
that  cannot  be  adapted  to  current  needs.  The  reason  is 
that  the  internal  data  management,  is  strongly  tied  to 
algorithms  and  computations. 

The  goal  of  the  Model-View-Controller  (MVC)  design 
pattern  is  -  in  addition  to  the  mentioned  separation  of  the 
UIMS  (“controller^  -  to  separate  the  application  object 
(“model”)  from  the  way  it  is  represented  to  the  user 
(“view”).  This  leads  to  a  system  structure  as  illustrated  in 
fig.  1. 


Figure  1:  Model- View-Controller  Design  Pattern 

Chapter  3.1  gives  an  example  of  a  program  that  follows 
the  MVC  design  concept.  In  a  more  general  view,  the 
MVC  concept  means,  that  in  modern  workstations  which 
are  normally  used  as  front-end  in  CCIS,  the  data  storage 
should  be  separated  from  the  applied  programs  and  tools. 
This  can  be  done  when  powerful  database  management 
systems  are  available  in  a  client  server  environment. 
When  data  storage  and  data  processing  capabilities  are 
separated,  user-  and  task-oriented  systems  can  be  devel¬ 
oped  that  control  the  information  flow  between  database 
and  front-end  tool. 

This  resulting  flexibility  in  the  construction  of  appropri¬ 
ate  user  interfaces  will  be  demonstrated  by  a  case  study 
which  is  the  topic  of  the  next  chapter. 
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3  Design  of  Human-Machine  Inter¬ 
faces  based  on  Componentware 
Technology 

The  following  case  study  demonstrates  the  benefits  of 
the  use  of  componentware  technology  which  was  intro¬ 
duced  in  chapter  2. 

3.1  Scenario 

In  a  former  military  conflict  it  was  necessary  to  maintain 
a  list  of  mine  accidents  which  happened  during  the  crisis. 
The  task  was  to  provide  the  appropriate  information  in 
an  event  list  which  was  part  of  a  situation  report.  This 
report  was  generated  as  a  series  of  PowerPoint  slides. 
Therefore  it  was  decided  to  maintain  the  event  list  di¬ 
rectly  as  a  table  in  the  presentation  program. 

Obviously,  this  way  of  “data  managing”  violated  the 
MYC  design  pattern  and  comprised  a  lot  of  disadvan¬ 
tages.  Problems  occurred  in  the  handling  of  the  data 
(almost  manual  processing  and  formatting)  and  last  but 
not  least  the  data  were  not  available  for  further  use  in  the 
command  and  control  process. 

The  current  situation  illustrated  by  this  scenario  empha¬ 
sizes  the  necessity  to  conduct  research  on  the  scientific 
conception  of  improved  task  and  user  adaptable  support 
systems  based  on  systematic  task  and  system  analyses. 
Two  examples  will  be  presented  below. 

3.2  Application  example  (1) 

The  componentware  approach  is  not  only  applicable  to 
improve  the  architecture  of  a  total  CCIS  (macro-view).  It 
can  also  be  used  to  construct  concrete  military  applica¬ 
tion  software  (micro- view).  The  following  is  an  example 
of  such  an  improved  military-off-the-shelf  product 
(MOTS). 


To  study  the  impact  of  componentware  architecture  a 
military  situation  display  system  has  been  modified  and 
implemented  as  a  component.  The  complete  human 
computer  interface  is  now  made  up  of  commercial-off- 
the-shelf  products  (COTS),  like  word  processor,  presen¬ 
tation  tool,  image  editor,  and  specialized  elements,  like 
situation  editor  (Kaster  1998)  as  well  as  geographic 
vector  and  raster  map  display  and  creation  programs. 

All  these  components  can  be  put  together  at  runtime 
according  to  die  actual  operational  requirements.  To 
achieve  a  smooth  exchangeability  of  the  specialized 
component  elements  the  underlying  software  design  is 
structured  by  strict  separation  of  the  user  interface  from 
the  remaining  system  functionality.  This  separation 
means,  e.g.,  that  dialogs  are  implemented  as  independent 
components  as  well  as  menu  bars.  They  do  not  belong  to 
the  kernel  of  the  system.  Therefore  they  can  be  easily 
replaced,  even  by  third  party  components  (e.g.,  a  new 
color  selection  technique). 

Furthermore  well  known  design  patterns  (Gamma  et.  al. 
1995)  such  as  Model-View-Controller  (MVC)  paradigm, 
Presentation-Abstraction-Controller  (PVC),  and  the 
Factory  pattern  have  been  used  to  group  the  parts  logi¬ 
cally.  Because  of  the  underlying  structure  of  the  MVC 
concept  the  user  can  choose  between  different  means  for 
presenting  information,  like  graphics  with  or  without 
geographic  background,  textual  output  of  object  struc¬ 
tures  and  attributes  (fig.  2),  and  for  manipulating  input 
data. 

3.3  Application  example  (2) 

Within  the  existing  client/server  infrastructure  an  alter¬ 
native  concept  has  been  developed  on  the  basis  of  a 
professional  database  management  system.  According  to 
the  requirements  of  the  afore  mentioned  MVC  pattern 
results  from  the  analysis  of  the  application  domain  were 
transformed  into  a  database  design  (model)  and  an  appli¬ 
cation  design  (view).  While  the 
underlying  development  process  is 
described  in  chapter  4  the  resulting 
application  components  and  user 
interfaces  are  illustrated  below. 

Fig.  3  shows  a  standard  input  mask 
for  mine  incidents  with  sophisti¬ 
cated  assistance  for  filling  in  the 
form,  i.e.,  entering  values  for  pa¬ 
rameters  and  attributes  describing 
an  incident..  Fig.  4  contains  several 
different  views  on  the  same  data 
which  are  designed  for  the  execu¬ 
tive  level  of  users.  Furthermore,  in 
the  componentware  environment  it 
is  possible  to  exchange  data  with 
other  presentation  tools  (e.g.,  pre¬ 
formatted  Excel  spreadsheets, 
PowerPoint  slideshows). 


Figure  2:  Situation  data  presented  in  a  multiple  windows  environment. 
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4  Rapid  Application  Development  of 
CCIS  based  on  CASE  Technology 

The  advantage  of  components  is  their  universality.  How¬ 
ever,  it  has  to  be  taken  into  account  that  this  flexibility 
may  cause  problems  in  the  management  of  the  total 
system.  This  is  especially  true  in  military  CCIS  of  high 
complexity.  Such  systems  comprehend  a  high  quantity  of 
data  and  often  show  a  wide  variety  of  heterogeneous 
data.  In  addition,  there  are  various  prerequisites  accord¬ 
ing  to  the  different  user  groups  for  the  interaction  with 
the  data.  There  are  requirements  for  inputting  and  ma¬ 
nipulating  as  well  as  for  analyzing,  evaluating  and  pre¬ 
senting  the  data.  There  is,  after  all,  the  necessity  for 
making  important  decisions  in  short  time  based  on  pre¬ 
sented  information.  The  aforementioned  characteristics 
require  especially  that  human  factors  principles  have  to 
be  incorporated  in  the  development  of  the  user  interface. 

As  mentioned  before,  a  well  defined  framework  needs  to 
be  defined,  so  that  independent  components  can  be  em¬ 
bedded  and  cooperate  as  generic  parts  of  a  system. 
Within  this  framework,  is  may  be  necessary  to  modify 
components  in  a  short  time.  Rapid  Application  Devel¬ 
opment  (RAD)  techniques  allow  an  evolutionary  imple¬ 
mentation  and  optimization  of  components.  This  will  be 
demonstrated  for  typical  database  applications. 

The  approach  described  herein  deals  with  the  application 
of  integrated  CASE  technology  (iCASE)1  (Hoppe  and 
Mempel  1998,  Kurbel  et  al.  1994)  in  the  development  of 
ergonomically  designed  user  interface  components  for  a 
database  of  a  CCIS  (Raster  et.al.  1999).  The  technical 
platform  is  the  Oracle  Developer  Suite. 

4.1  CASE  Technology  -  Bridging  the  Gap  between 
User  Requirements  and  Information  Systems 

The  architecture  of  modem  information  systems  is  char¬ 
acterized  by  a  decentral  processing  along  with  central 
administration  of  information.  In  order  to  use  the  data 
interactively  within  this  client/server  architecture  con¬ 
sistent  and  transparent  interfaces  have  to  be  developed. 
These  enable  task  and  situation  dependent  access,  input 
and  recherche  of  database  information. 

Figure  5  illustrates  a  general  procedure  to  design  a  sys¬ 
tem  using  CASE  technology.  Based  on  the  analysis  of 
user  requirements  the  application  of  CASE  enables  an 
almost  fully  automated  and  standardized  creation  of  user 
forms  for  accessing  the  database.  The  application  do¬ 
main  is  structured  in  organizational  items  and  processes 
(business  process  model).  In  the  following  system  mod¬ 
eling  phase  information  objects  of  the  process  model  are 
represented  in  an  entity  relationship  diagram  (ERD)  and 
functions  are  defined  in  a  function  hierarchy  diagram 
(FHD).  In  using  so-called  wizards  the  application  soft¬ 
ware  is  nearly  automatically  developed.  The  entities  are 
transformed  into  tables  of  the  database  (database  design) 


1  iCASE  =  integrated  Computer  Aided  Systems  Engineering 


and  the  functions  are  transformed  into  the  application 
design. 

At  this  point  available  techniques  of  CASE  were  utilized 
to  incorporate  ergonomic  principles  in  the  design  proc¬ 
ess  of  the  application  user  interfaces.  Oracle  CASE  of¬ 
fers  the  use  of  libraries ,  where  the  functionality  of  dif¬ 
ferent  forms  is  defined  and  stored  in  form  of  centralized, 
reusable  and  consistent  code.  It  allows  the  definition  of 
preferences ,  where  general  adjustments  about  layout, 
e.g.  scrollbar  position,  take  place.  Furthermore,  tem¬ 
plates  are  used,  e.g.  to  define  the  exact  information  about 
the  layout  of  the  header  or  footer  or  the  button  palette. 
Last  but  not  least,  reusable  master  forms  can  be  defined, 
in  order  to  avoid  redundancy  and  to  assure  consistent 
behavior  of  several  application  forms  in  an  object- 
oriented  manner. 

Using  the  description  of  the  application  domain  in  the 
repository  and  the  predefined  dialog  components  as  well 
as  presentation  rules  a  consistent  layout  of  the  user  inter¬ 
faces  is  created.  Integrated  support  functions  enable  user 
guidance  and  user  assistance  in  operating  the  application 
forms,  e.g.  semi-automated  filling  of  the  masks,  locking 
fields  depending  on  user  input,  offering  list  of  values  for 
valid  input  as  well  as  online  help.  The  rapid  application 
development  supported  by  CASE  allows  an  evolutionary 
development  process  and  gives  the  chance  for  early  user 
feedback. 

4.2  Application  of  CASE 

The  heterogeneous  data-sets  in  military  command  and 
control  systems  ask  for  various  views  on  the  data  in 
order  to  satisfy  user  needs  as  well  as  to  fulfill  task  re¬ 
quirements.  Consequently,  the  visualization  of  the  data 
has  been  conducted  in  various  manners  (Fig.  3  and  4). 

(1)  Using  the  CASE  technology  the  data  were  visual¬ 
ized  by  several  Oracle  database  tools.  FORMS  en¬ 
ables  interaction  in  tabular  form,  GRAPHICS  sup¬ 
ports  pictorial  representation,  and  REPORTS  gener¬ 
ates  written  protocols. 

(2)  According  to  user  requests  the  forms  were  con¬ 
nected  with  Microsoft  Office  tools.  Forms  with  free 
text  were  coupled  with  WORD,  tabular  data  were 
coupled  with  EXCEL,  single  documents  can  be 
transformed  to  PowerPoint  in  order  to  prepare  pres¬ 
entations.  In  this  manner  users  can  work  on  their 
data  in  familiar  environments. 

(3)  Another  connection  took  place  with  a  situation  dis¬ 
play  program.  Here,  the  forms  are  displayed  as  dia¬ 
log  components  on  digital  maps,  where  interaction 
results  in  the  modification  of  the  situation  repre¬ 
sented  on  the  map. 

(4)  In  order  to  fulfill  the  requirements  for  an  universal 
user  interface  the  presentation  of  the  database  in¬ 
formation  can  be  presented  based  on  Internet  tech¬ 
nology.  The  user  can  access  the  database  via  a 
commercial  web  browser. 
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Figure  5:  System  design  with  iCASE  technology  enables  the  implicit  incorporation  of  ergonomic  principles 


5  Conclusions 

Current  problems  in  the  use  of  CCIS  are  often  caused  by 
insufficiencies  of  the  human-machine  interface.  In  order 
to  overcome  these  problems  it  is  necessary  to  take  notice 
of  strategies  which  help  to  create  task  oriented  user  inter¬ 
faces  that  fulfill  the  requirements  specified  in  ergonomic 
standards  (ISO  1995,  VDI  1990). 

Componentware  approaches  provide  means  that  support 
the  practical  realization  of  systems  based  on  precise 
theoretical  concepts.  Design  patterns  help  to  structure  the 
application  domain  by  delivering  ready-made  design 
solutions. 

Besides  theoretical  aspects,  toolkits  are  required  that 
support  complex  development  processes.  In  this  paper 
Rapid  Application  Development  was  referred  to  as  a  way 
that  provides  solutions  in  a  short  time.  This  method  al¬ 
lows  early  user  participation  which  is  extremely  impor¬ 
tant  from  a  human  factors  point  of  view.  The  user  inter¬ 
face  for  the  access  to  the  heterogeneous  data  can  be 
created  by  using  iCASE  technology.  Such  a  platform  can 
provide  the  basic  means  to  explicitly  incorporate  ergo¬ 
nomic  knowledge  in  the  design  process. 

The  paper  presented  examples  for  benefits  of  compo¬ 
nentware  technology  in  a  macro-view  (considering  a 
system  as  a  whole)  as  well  as  in  a  micro- view  (designing 
specific  MOTS  products). 

In  the  research  project  it  was  demonstrated  that  the  ap¬ 
plication  of  componentware  allows  the  creation  of  inter¬ 
faces  in  individual  user  environments  in  a  consistent, 
complete  and  standard  conform  manner.  The  use  of 
Internet  technology  will  strengthen  this  effect. 
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Quelle  Assistance  pour  la  Gestion  des  Missions  Tactiques  ? 
A  Propos  du  Projet  Copilote  Electronique 
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Institut  de  Medecine  Aerospatiale  du  Service  de  Sante  des  Armees 
BP73,  91223  Bretigny-sur-Orge  Cedex 
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Summary.  Aircraft  automation  has  forced 
designers  to  enhance  their  systems  with  aids  to  keep 
the  pilot  "in  the  loop".  The  development  of 
assistance  systems  are  in  line  with  the  evolution  of 
"human  factor"  knowledge.  Drawing  from  the 
concept  of  human-centered  design,  we  introduce 
new  assistance  systems  based  upon  Man-Machine- 
Environment  interactions.  Applied  to  fighter 
aircraft,  these  concepts  gave  birth  to  the  "Electronic 
Copilot"  program.  The  Electronic  Copilot  is  a 
cooperative  aid.  It  is  aimed  at  helping  the  pilot 
operate  at  his  best  cognitive  compromise.  The 
philosophy  and  the  Electronic  Copilot  program 
itself  are  described  in  the  paper.  The  last  part  is 
dedicated  to  the  lessons  to  be  drawn  from  the 
program.  Beyond  any  technical  issues  and  the 
challenge  of  designing  with  Knowledge  based 
systems  and  Artificial  Intelligence  technology,  the 
assistance  principles  evaluated  in  this  study  appear 
to  be  promising  if  humans  are  ever  to  stay  in  the 
loop  of  complex  system  control. 

1.  INTRODUCTION 

Les  aeronefs  de  derniere  generation  offrent  grace  a 
leurs  capacites  manoeuvrieres  et  a  leurs  systemes 
embarques  plus  de  possibility  d'emploi  que  leurs 
predecesseurs.  L’equipage  dispose  continuellement 
d'un  flux  d’information  qu’il  doit  selectionner  et 
analyser.  Les  missions  s'effectuent  dans  des  cadres 
tactiques  ou  l'aeronef  n'est  jamais  seul.  Les 
decisions  doivent  tenir  compte  d’un  dispositif  global 
qui  prend  en  compte  les  composantes  aeriennes, 
terrestres  voire  navales  qu'elles  soient  amies  et 
ennemies.  Les  contraintes  temporelles  sont  fortes  et 
pesent  a  tout  moment  sur  les  mecanismes  de 
perception,  de  comprehension  et  de  prise  de 
decision.  Face  a  cette  complexity,  aider  1’equipage 
est  le  souci  permanent  des  concepteurs.  Si  Ton 
compare  le  cockpit  d’un  avion  de  combat  actuel  a 
celui  d’un  avion  des  annees  70,  les  evolutions  sont 
evidentes  :  les  sources  d’informations  se  sont 
multiplies,  les  informations  sont  de  plus  en  plus 
symboliques,  les  moyens  de  communication 
homme-machine  utilisent  de  plus  en  plus  de 


modalites  et  des  taches  entieres  sont  deleguees  a 
des  automatismes. 

Malgre  ces  evolutions,  l'equipage  est  toujours 
considere  comme  le  facteur  limitant  de  la 
performance  homme-machine.  Devant  ce  constat,  la 
notion  d'aide  et  d’assistance  a  l’equipage  est 
essentielle  dans  le  processus  de  conception  des 
futurs  aeronefs.  Quelle  aide  doit-on  apporter  a 
l’equipage  pour  qu’il  reste  maitre  de  la  situation  ? 
Les  principes  d’elaboration  des  aides  reposent  sur 
differents  modeles  des  relations  homme-machine. 
Force  est  de  constater  que  pour  depasser  les  limites 
des  aides  actuelles  et  envisager  les  futurs  systemes 
homme-machine,  il  faut  mieux  prendre  en  compte 
la  dynamique  de  la  cognition  humaine  en  situation 
de  travail.  A  travers  cet  article,  on  se  propose  de 
presenter  les  principes  d’elaboration  d'une  aide 
"intelligente"  au  pilotage  pour  avion  d'armes  avant 
de  presenter  les  enseignements  d'une  etude  portant 
sur  la  conception  d'une  assistance  au  pilotage,  le 
programme  "Copilote  electronique".  Realise  dans  le 
cadre  d’un  developpement  exploratoire  soutenu  par 
la  Delegation  Generate  de  l'Armement  du  Ministere 
de  la  Defense,  ce  programme  a  rassemble  5 
industriels  de  1'aeronautique  (Dassault-Aviation, 
Dassault  Electronique,  Matra,  Sagem  et  Sextant- 
Avionique)  et  l’lnstitut  de  Medecine  Aerospatiale 
du  Service  de  Sante  des  Armees  (IMASSA).  Seuls 
les  aspects  propres  aux  facteurs  humains  et  sous  la 
responsabilite  de  1'IMASSA  seront  traites  dans  cet 
article. 


2.  QUELLES  AIDES  POUR  L’OPERATEUR  ? 

2.1.  D’une  approche  techno-centree  aux  approches 
centrees  sur  l’utilisateur 

Pour  aider  le  pilote  a  repondre  aux  contraintes  des 
aeronefs  de  derniere  generation,  il  est  interessant 
d'analyser  1'evolution  des  principes  de  conception 
des  aides  existantes.  L'approche  techno-centree  de 
la  conception  des  cockpits  qui  predomine  depuis  les 
annees  70  s'est  rapidement  contentee  d'une  vision 
restrictive  de  I’homme  au  travail  pour  ne  souligner 
que  ses  limitations.  En  consequence,  il  faut 
remplacer  l'homme  des  qu’on  le  peut  par  des 


Paper  presented  at  the  RTO  HFM  Symposium  on  “  Usability  of  Information  in  Battle  Management 
Operations' \  held  in  Oslo ,  Norway,  10-13  April  2000,  and  published  in  RTO  MP-57. 
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systemes  automatises  qui  sont  plus  fiables.  Mais  la 
difficulte  vient  du  fait  qu'on  ne  sait  pas  totalement 
suppleer  l'etre  humain,  en  particulier  pour  les  taches 
intellectuelles  de  haut  niveau  ou  adaptation  et 
creativite  sont  les  garants  de  la  securite  et  de  la 
performance.  II  en  resulte  que  les  aides  apportees 
sont  plus  guidees  par  le  savoir-faire  technologique 
que  par  les  besoins  de  l’operateur.  Sur  un  plan 
pratique,  on  automatise  ce  que  Ton  sait  faire  et  a 
charge  pour  l'operateur  d'assurer  la  coherence  de 
I'ensemble  (principe  du  "left-over”).  Cest  ainsi  que 
sont  rapidement  apparus  certains  paradoxes  de 
l’automatisation  : 

-  Perte  de  competence  due  au  manque  de 
pratique  pour  les  taches  realisees  par  les  aides  (1) ; 

-  Mauvaise  representation  du 
fonctionnement  des  aides  et  done  difficulte  a 
comprendre  la  situation  (2) ; 

-  Apparition  de  nouvelles  erreurs  dues  aux 
aides  (3) ; 

-  Necessite  encore  plus  grande  de 
l'operateur  lorsque  le  systeme  d’aide  est  defaillant 

(4) ; 

-  En  reduisant  la  charge  de  travail,  l'aide 
induit  pour  l’operateur  un  etat  de  sous-charge  qui 
est  responsable  d’une  perte  d’efficacite  lorsque 
survient  une  situation  critique  (5). 

11  ressort  de  ces  paradoxes  le  besoin  d'une  meilleure 
comprehension  des  regies  de  couplage  entre 
Thomme  et  la  machine  pour  definir  une  conception 
veritablement  centree  sur  l'operateur.  On  ne  peut 
plus  se  contenter  d'un  partage  des  taches  en 
fonction  des  points  forts  de  I'homme  ou  des 
automatismes  comme  le  preconisait  Fitts  (6). 
L'allocation  de  fonctions  entre  I'homme  et  la 
machine  doit  etre  basee  sur  un  principe  de 
complementarity  et  non  de  competition  (7).  Plutot 
que  de  se  baser  sur  les  limitations  des  composants 
du  systeme,  le  principe  est  de  voir  comment  les 
hommes  et  les  machines  peuvent  se  completer  et 
s'aider  mutueilement.  Les  fonctions  des  hommes  et 
des  machines  sont  mutueilement  dependantes  et 
necessaires  pour  realiser  le  but  (8).  De  plus,  cette 
allocation  doit  etre  dynamique  pour  permettre  la 
flexibility  du  systeme  socio-technique  (9).  Une 
allocation  trop  statique  va  rendre  difficile,  voire 
impossible  la  flexibilite  humaine  qui  permet  a 
l'operateur  de  s’adapter  aux  contraintes  des 
situations  en  fonction  de  ses  propres  ressources. 

2.2.  Les  approches  centrees  sur  Thomme 
Depuis  1'emergence  des  facteurs  humains,  la 
conception  centree  sur  l'operateur  a  pris  plusieurs 
formes.  Derriere  ces  differentes  approches  se 
cachent  des  modeles  du  fonctionnement  humain.  Le 
modele  ie  plus  ancien  est  de  considerer  l'etre 
humain  comme  un  systeme  de  traitement  de 
l'information  dont  les  capacites  dependent  des 
capacites  et  limites  des  sous-systemes  qui  le 


composent.  L’aide  que  Ton  apporte  a  l'operateur 
vise  alors  a  optimiser  les  capacites  de  chacun  des 
sous-systemes  (par  exemple,  le  systeme  visuel  ou 
les  capacites  de  memorisation).  Fortes  de  ce 
courant,  de  nombreuses  ameliorations  ont  pu  etre 
apportees  dans  la  conception  des  interfaces  pour 
aider  les  operateurs  a  mieux  interagir  avec  la 
situation  de  travail.  Cependant  une  limite  de  cette 
approche  est  rapidement  apparue  avec  la  necessite 
pour  les  operateurs  d'utiliser  simultanement 
plusieurs  de  leurs  sous-systemes,  par  exemple  lire 
une  information  sur  un  ecran  et  retenir  l’information 
pergue.  Dans  ce  cas,  connaitre  les  caracteristiques 
de  la  perception  visuelle  et  de  la  memoire  humaine 
ne  suffit  plus  a  decrire  et  expliquer  le 
comportement  humain  en  situation  dynamique. 

Un  autre  modele  complementaire  du  precedent  peut 
alors  etre  utilise,  celui  des  ressources 
attentionnelles  (10).  L’attention  est  1’application 
volontaire  de  l’esprit  a  un  objet  precis,  elle  est  une 
instance  de  controle  et  d’orientation  de  I'activite 
(11).  Les  ressources  attentionnelles  sont  limitees  et 
ne  permettent  pas  de  tout  faire  simultanement  et  de 
fagon  exhaustive.  L'operateur  doit  orienter  son 
activite  en  fonction  des  caracteristiques  de  la 
situation  et  de  ses  intentions.  Le  principe  des  aides 
est  alors  de  permettre  a  l'operateur  de  faire  ou  de 
controler  le  plus  de  choses  possibles  sans  que  ses 
activites  ne  depassent  ses  propres  capacites 
attentionnelles.  Deux  voies  sont  possibles  pour 
aider  l'operateur  : 

-  Instaurer  une  etape  integrative  entre  les 
elements  bruts  de  la  situation  et  reformation 
presentee  a  l'operateur.  L'aide  consiste  a  donner  aux 
informations  un  contenu  symbolique  plus  important 
pour  faciliter  les  haut s  niveaux  de  traitement.  L'aide 
peut  aussi  consister  a  faciliter  la  disponibilite  de 
l'information  en  fonction  de  la  situation  et  des 
contraintes  de  l'operateur.  Dans  ce  cas,  la  difficulte 
reside  dans  l'elaboration  d'un  modele  predictif  de 
I'activite.  L'absence  de  prise  en  compte  de  la 
variability  interindividuelle,  de  la  gestion  des 
evenements  non  prevus  et  de  I'activite  reelle  des 
operateurs  n’a  pas  permis  de  proposer  des  aides 
concluantes  a  partir  de  modeles  normatifs  de  la 
tache  (12).  Cependant  ce  principe  d’aide  est 
toujours  d’actualite  a  travers  la  notion  d'aide 
adaptative  ("adaptive  aids"  pour  les  anglo-saxons) 
mais  il  faut  alors  le  recentrer  sur  l'analyse  de 
l’interaction  entre  l’operateur  et  1’environnement. 

-  Decharger  l’operateur  d'une  partie  de  son 
activite.  Cette  voie  est  celle  de  l'allocation  des 
fonctions  entre  l'operateur  et  l’aide.  Le  principe 
d'allocation  complementaire  est  maintenant  admis 
et  de  nombreux  travaux  vont  dans  ce  sens  (13,14). 
On  parle  d’allocation  "complementaire"  dans  la 
mesure  ou  l'operateur  humain  et  l'aide  constituent 
un  systeme  cognitif  distribue  dans  lequel  les 
mecanismes  de  traitement  de  l’information  de 
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l'operateur  sont  en  equilibre  dynamique  avec  l'aide. 
L'allocation  est  alors  dynamique  et  depend 
fortement  de  1'interaction  entre  l’operateur  et  la 
situation.  L’ensemble  des  travaux  sur  le  partage  des 
taches  montre  que  la  delegation  d’une  tache  a  une 
aide  ne  peut  jamais  etre  totale  pour  l’operateur. 
L'operateur  est  toujours  le  decideur  final  et  au 
minimum  il  doit  s'assurer  du  bon  fonctionnement  de 
l’aide  (15).  Face  a  une  aide,  l'etre  humain  n'est  pas 
un  operateur  passif  mais  bien  le  superviseur  d'un 
systeme  automatise  subordonne.  Son  attitude  et  son 
comportement  dependent  de  la  representation  qu'il  a 
des  competences  de  l'aide  et  de  la  confiance  qu'il 
peut  lui  accorder. 

Une  autre  vision  de  1'homme  au  travail  pour 
concevoir  des  aides  est  celle  decrite  par  les 
approches  de  la  cognition  "situee".  L'activite 
humaine  vise  en  permanence  a  assurer  un 
compromis  cognitif  entre  les  objectifs  a  atteindre, 
les  risques  d’erreurs  et  la  controle  de  la  fatigue  ou 
du  stress  lie  a  la  realisation  du  travail.  L’adaptation 
a  la  complexity  des  situations  repose  sur  la  gestion 
d'une  prise  de  risque  constante  pour  realiser  la  tache 
a  un  cout  cognitif  acceptable.  L'operateur  gere  en 
permanence  son  activite  pour  degager  des  marges 
d’adaptation  qui  lorsqu'elles  sont  depassees,  sont 
sources  de  perte  de  controle  de  la  situation  ou 
d'erreur.  Mais  comme  le  decrit  Reason  (16),  en 
raison  de  leurs  natures  heuristiques,  les  compromis 
peuvent  generer  une  performance  qui  sans  etre 
optimale  est  acceptable,  voire  generer  des  erreurs 
lorsque  les  compromis  ne  sont  pas  adaptes  aux 
situations  rencontrees.  Dans  ce  contexte,  on  peut 
envisager  l'aide  comme  un  outil  d’aide  a  la  gestion 
du  compromis  cognitif  et  a  la  preservation  des 
marges  d'adaptation.  L'aide  n'est  plus  une  "prothese 
cognitive"  qui  supplee  aux  points  faibles  de 
l’operateur.  II  faut  faire  comprendre  a  l'operateur  un 
point  de  vue  ou  un  conseil  qu'il  connait  mais  auquel 
il  n'a  pas  pense.  L'aide  est  alors  une  veritable  aide  a 
la  comprehension,  a  l'anticipation  et  a  la  prise  de 
decision.  La  notion  d’aide  se  rapproche  ainsi  de 
celle  d'assistance  dans  la  mesure  oil  elle  est  en 
permanence  "pres"  de  l'operateur,  c’est-a-dire 
proche  de  ses  raisonnements.  On  ne  considere  plus 
l'aide  a  travers  l'allocation  de  fonctions  mais 
comme  un  systeme  cooperatif. 

3.  A  LA  BASE  DE  L'AIDE  :  L'ACTIVITE  DU 
PILOTE 

Concevoir  une  aide  ne  peut  s'envisager  que  si  Ton  a 
clairement  identifie  les  enjeux  cognitifs  du  pilote, 
c'est-a-dire  les  elements  qui  vont  participer  au 
reglage  du  compromis  cognitif  et  des  marges 
d’adaptation.  Les  travaux  que  nous  avons  menes  sur 
l’activite  de  pilotage  a  partir  d'observations  en  vol  et 
en  simulateur,  et  sur  la  base  d’entretiens  avec  les 


equipages  ont  permis  de  decrire  les  grandes  lignes 
d’un  modele  du  fonctionnement  cognitif  du  pilote 
(17,18).  Ces  caracteristiques  ne  sont  pas  specifiques 
a  l’aeronautique  de  combat  et  peuvent  se  rencontrer 
dans  d’autres  situations  de  controle  de  processus 
dynamique  et  complexe.  Dans  le  cadre  de  la 
conception  d’une  aide  cooperative,  il  est  important 
de  prendre  en  compte  les  points  suivants  : 

-  Pour  maitriser  la  complexity  des 
situations  de  travail  dynamiques,  la  nature  de 
l'activite  des  pilotes  est  essentiellement  anticipative. 
L'anticipation  permet  de  ne  pas  etre  surpris  par  un 
evenement  et  de  devenir  reactif,  mode  de 
fonctionnement  pour  lequel  1’homme  n'est  pas 
particulierement  doue,  ce  que  proposent  bien 
souvent  les  aides  "intelligentes"  existantes. 
L’anticipation  permet  d’eviter  les  situations  a 
probleme  en  changeant  le  cours  de  la  tache  pour 
rester  dans  ce  que  I'on  sait  faire  et  ce  que  Ton  a 
planifie  ; 

-  La  comprehension  et  la  decision  sont 
indissociables  du  choix  de  la  solution  et  de  la 
faisabilite  de  sa  realisation.  Cette  relation  tres  forte 
entre  decision  et  action  necessite  de  faire  en 
permanence  des  evaluations  sur  les  hypotheses 
alternatives  ; 

-  Les  pilotes  gerent  en  permanence 
plusieurs  horizons  temporels  qui  peuvent  amener  a 
des  niveaux  de  comprehension  et  de  decision 
differents  suivant  les  objectifs  a  court  ou  long 
terme.  Les  decisions  a  court  terme  ne  sont  pas 
toujours  les  plus  adaptees  au  long  terme.  Nombres 
de  reactions  prennent  en  compte  ce  double  horizon 
temporel  et  s'effectuent  en  deux  temps  avec  des 
objectifs  differents  a  chaque  fois  ; 

-  Les  pilotes  fonctionnent  a  differents 
niveaux  de  representation  et/ou  de  controle  suivant 
les  exigences  de  la  situation  et  leurs  propres 
connaissances.  Une  aide  doit  respecter  au  mieux  ses 
niveaux  de  fonctionnement.  Hollnagel  (19)  decrit 
ainsi  4  niveaux  de  controle  de  l’activite  : 
desorganise,  opportuniste,  tactique  et  strategique. 

-  Le  fonctionnement  des  pilotes  est  regi  par 
un  principe  d'economie  qui  vise  d'une  part,  a  rester 
dans  une  zone  de  confort  pour  ne  pas  etre  en 
permanence  au  maximum  de  ses  capacites,  ce  qui 
serait  generateur  de  fatigue  et  d'autre  part,  a  garder 
en  reserve  une  capacity  d'intervention  pour  faire 
face  aux  evenements  non  planifies.  La  consequence 
du  principe  d'economie  est  que  les  pilotes  peuvent 
admettre  de  comprendre  superficiellement  un 
aspect  de  la  situation  qu'ils  jugent  non  prioritaire 
voire  de  ne  pas  le  comprendre  du  tout  lorsqu'ils  le 
considerent  comme  peu  important.  Cela  s'oppose  a 
une  vision  d'un  pilote  qui  doit  tout  percevoir,  tout 
comprendre  avant  de  decider,  ce  qui  est  de  par  son 
cout  cognitif  peu  compatible  avec  les  exigences  des 
situations  rencontrees  ; 

-  Pour  rester  dans  un  registre  de 
fonctionnement  compatible  avec  la  sauvegarde  de 
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marges  d’adaptations,  le  pilote  accepte  en 
permanence  un  niveau  de  risque.  Cette  prise  de 
risque  s'effectue  par  une  reduction  de  la  complexity, 
une  conduite  par  anticipation  et  une  economie  des 
ressources  faisant  preferer  (quand  cela  est  possible) 
un  niveau  de  conduite  automatique  a  un  niveau  de 
conduite  plus  reflechi.  devaluation  du  risque  par  le 
pilote  s’effectue  a  deux  niveaux  :  le  risque  externe 
qui  correspond  au  risque  objectif  et  le  risque  interne 
qui  represente  le  sentiment  qu'a  le  pilote  de 
maitriser  ou  non  la  situation.  Le  sentiment  de  risque 
eleve  apparaTt  de  fa<?on  preferentielle  lorsque  le 
risque  interne  est  eleve.  Dans  ce  cas,  le  pilote 
abaisse  le  niveau  de  risque  en  preferant  abaisser  le 
risque  interne  aux  depens  du  risque  externe.  Cela 
explique  I’echec  de  nombreuses  aides  qui  basent 
leur  assistance  sur  le  risque  externe. 

-  La  recuperation  des  erreurs  par  le  pilote 
est  un  mecanisme  important  de  regulation  de  son 
activite  et  de  fiabilite  humaine.  Les  meilleurs 
pilotes  sont  ceux  qui  font  le  moins  d'erreurs  mais 
aussi  ceux  qui  savent  detecter  et  recuperer  les 
erreurs  commises.  Cette  recuperation  n’est  possible 
que  si  le  pilote  dispose  de  marges  cognitives 
suffisantes  pour  effectuer  cette  recuperation.  Par 
ailleurs,  les  erreurs  renvoient  au  pilote  une  image 
de  son  propre  fonctionnement  qui  lui  permet  en 
retour  de  mieux  regler  le  compromis  cognitif  en 
termes  de  risque-performance. 


4.  LE  PROGRAMME  COPILOTE 
ELECTRONIQUE 

La  philosophic  du  "copilote  electronique"  est  de 
proposer  au  pilote  une  assistance  tout  comme 
pourrait  le  faire  un  membre  d'equipage  assis  a  cote 
de  lui  dans  le  cockpit,  d’ou  son  nom.  Cette 
assistance  a  pour  objectifs  : 

-  De  ne  pas  se  priver  des  defenses 
naturelles  du  pilote  et  de  ne  pas  les  contrarier  ; 

-  De  permettre  au  pilote  de  regler  son 
compromis  cognitif  en  lui  laissant  le  controle  de  la 
situation  et  de  la  prise  de  risques,  tout  en  favorisant 
la  visibility  de  ses  actions  et  des  actions  des 
systemes. 

Le  Copilote  Electronique  est  un  systeme  a  base  de 
connaissances  expertes  utilisant  les  techniques  de 
l'intelligence  artificielle.  Les  premieres  phases  du 
programme  avaient  pour  objectif  de  demontrer  la 
faisabilite  et  la  pertinence  d’une  telle  assistance  et 
ont  abouti  a  la  realisation  d’une  maquette  de 
demonstration  sans  contrainte  "temps  reel". 
L'originalite  de  cette  etude  a  ete  d’integrer  des  les 
phases  initiales  de  la  conception  des 
recommandations  "facteurs  humains"  pour  la 
definition  des  fonctionnalites  et  de  1’architecture  de 
l'aide. 

"Copilote  electronique"  est  une  aide  globale  qui 
integre  l’ensemble  des  elements  de  la  mission  :  le 


pilote,  l’aeronef,  l’environnement  tactique  et  la 
cadre  de  mission  (Figure  1).  Ce  choix  resulte  des 
etudes  preparatories  qui  avaient  montre  que  la 
mission  est  un  tout  pour  le  pilote  et  qu’il  n'est  pas 
possible  d'isoler  differents  objets  si  Ton  veut 
respecter  la  gestion  des  compromis  cognitifs.  En 
consequence,  on  ne  peut  envisager  la  gestion 
tactique  sans  prendre  en  compte  les  contraintes 
imposees  par  d’autres  objets  comme  la  navigation 
ou  I'etat  de  la  cellule  et  des  systemes. 

Les  informations  disponibles  dans  l'aeronef  (base 
de  donnees,  capteurs,  calculateurs,  actions  pilote, 
etats  des  systemes)  permettent  de  disposer  des 
parametres  d'ou  sont  inferees  les  informations 
symboliques  propres  a  chaque  composante 
fonctionnelle.  Une  etape  integrative  gere  les 
differentes  contraintes  propres  a  chaque 
composante  en  adequation  avec  un  modele  de 
reconnaissance  des  intentions  du  pilote.  Les 
conseils  et  analyses  elabores  sont  alors  dependants 
de  fanalyse  contextuelle  et  des  compromis  cognitifs 
du  pilote.  La  derniere  etape  est  celle  de  l’interface 
homme-machine. 
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Figure  1.  Architecture  du  Copilote  Electronique 

4.1.  Les  principes  ergonomiques 
L’aide  doit  repondre  a  4  principes  ergonomiques 
pour  etre  en  adequation  avec  le  fonctionnement 
cognitif  du  pilote  (20) : 

-  Donner  au  pilote  un  niveau  de 
comprehension  qui  lui  permet  de  comprendre  mais 
aussi  de  critiquer  les  propositions  entries  par  l’aide. 
Derriere  cette  recommandation  apparait  la  crainte 
pour  le  pilote  d’une  solution  "magique"  tombee  du 
ciel  dont  il  ne  verrait  pas  immediatement  les  tenants 
et  les  aboutissants.  Accepter  une  solution  magique, 
c'est  donner  la  totale  main  mise  de  l’aide  sur  le 
pilote,  chose  qui  n’est  pas  acceptable  au  regard  des 
enjeux  sous-jacents.  On  identifie  4  niveaux  de 
comprehension  pour  garantir  le  couplage  de  l’aide  : 
comprendre  la  proposition,  etre  capable  de 
I’evaluer,  comprendre  comment  le  systeme  l'a 
elaboree  et,  etre  capable  de  la  mettre  en  oeuvre. 
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C'e st  a  partir  de  ces  niveaux  de  comprehension  que 
le  pilote  pourra  se  construire  une  representation 
mentale  des  competences  de  l'aide.  L'interet  de  ces 
niveaux  de  comprehension  est  naturellement  de 
maintenir  I'operateur  dans  la  boucle  de  controle  de 
la  situation  et  de  lui  permettre  d'avoir  confiance 
dans  l’aide.  Dans  ce  but,  Billings  et  Woods  (21) 
vont  jusqu’a  preconiser  que  l’assistance  puisse 
communiquer  a  I'operateur  ses  intentions  et  ce 
d’autant  plus  que  les  enjeux  sont  eleves.  Le 
fonctionnement  de  l’aide  doit  pouvoir  etre  predit  par 
I'operateur  pour  que  ce  dernier  Tutilise  au  mieux  ; 

-  L'aide  doit  prendre  en  compte  les 
variability  interindividuelles.  L'analyse  et/ou  la 
solution  de  pilotes  experimentes  ne  sont  pas 
forcement  les  memes  que  celles  de  pilotes  tout  juste 
qualifies.  Des  differences  existent  dans  la 
disponibilite  en  memoire  des  informations  pour 
comparer  les  situations  rencontrees  aux  situations 
vecues  mais  aussi  dans  la  capacite  a  disposer 
d’habiletes  sensori-motrices  et  cognitives  pour 
realiser  des  solutions  envisagees  ; 

-  La  coherence  de  l'expertise  est  un  point 
essentiel  pour  l’elaboration  de  la  confiance.  Cette 
recommandation  s’oppose  aux  expertises 
"patchwork”  que  Ton  rencontre  dans  beaucoup  de 
systemes  experts.  II  est  important  que  l'expertise  ne 
soit  pas  un  amalgame  d'expertise  qui  ne 
corresponde  a  aucune  expertise  humaine.  Le  pilote 
attend  d'une  aide  un  fonctionnement  coherent  c’est- 
a-dire  predictible  et  sur  lequel  il  pourra  compter  : 
savoir  que  le  systeme  lui  fera  penser  a  une  analyse 
ou  a  une  solution  a  laquelie  il  n'aura  pas  pense, 
mais  qui  est  compatible  avec  ses  savoirs  et  savoir- 
faire  ; 

-  Adapter  le  conseil  et  le  dialogue  homme- 
machine  aux  capacites  de  I'operateur  en  fonction 
des  exigences  de  la  situation.  Plusieurs  niveaux 
d'interaction  peuvent  etre  decrits  allant 
d'interactions  directives  tant  d'un  point  de  vue 
semantique  que  syntaxique  dans  les  situations 
d'urgence  a  faible  delai  temporel,  aux  interactions 
plus  eiaborees  lorsque  les  marges  temporelles  sont 
plus  importantes. 

Ces  exigences  ergonomiques  ont  2  consequences 
pour  la  realisation  du  Copilote  Electronique  : 

-  Pour  etre  proche  des  pilotes,  les  aides 
doivent  integrer  dans  leur  architecture  les 
specificites  du  fonctionnement  cognitif  des  pilotes 
et  non  plus  se  contenter  d'une  ressemblance  de 
"surface"  au  niveau  des  interfaces  ; 

-  La  programmation  des  aides  ne  peut  se 
faire  qu'en  integrant  de  l'expertise  humaine  sous  la 
forme  de  bases  de  connaissances. 

4.2.  Les  actions  facteurs  humains 
A  cote  de  la  definition  de  ces  principes 
ergonomiques,  4  actions  facteurs  humains  ont  ete 
conduites  :  definition  des  fonctionnalites  de  l'aide, 


constitution  de  la  base  de  connaissances,  definition 
des  principes  de  dialogue  homme-machine  et 
evaluation  du  concept  "copilote  electronique". 

Les  2  premieres  actions  se  sont  construites  autour 
du  recueil  de  l'expertise  operationnelle.  Une 
methode  detraction  et  de  formalisation  commune 
aux  differents  partenaires  de  l'etude  a  ete  elaboree. 
L’objectif  de  cette  methode  etait  de  recueillir  de 
fagon  exhaustive  aupres  de  pilotes  operationnels  en 
escadron  de  chasse  une  expertise  coherente  sur  les 
savoirs  et  savoir-faire  en  usage  afm  de  constituer  la 
base  de  connaissances  du  Copilote  Electronique. 
Pour  favoriser  la  verbalisation  des  connaissances, 
des  entretiens  construits  autour  de  simulations 
papier-crayon  ont  ete  menes.  Une  extraction  des 
connaissances  en  profondeur  avec  2  pilotes  (fun  de 
Defense  Aerienne,  l’autre  de  Penetration  Basse 
Altitude)  a  ete  preferee  a  une  approche  multi¬ 
experts.  Les  verbalisations  recueillies,  environ  50 
heures  d’entretien  avec  chaque  pilote,  ont  ete 
transcrites  sur  papier  avant  d'etre  formalisees.  Le 
formalisme,  specialement  developpe  pour  l'etude 
ainsi  qu'un  outil  de  formalisation  X-Pert,  se 
decompose  en  4  concepts  :  objets,  proprietes, 
actions  et  raisonnements.  Le  formalisme  a  ete 
congu  pour,  d'une  part  pouvoir  representer 
1’ensemble  des  connaissances  verbalisees  et,  d'autre 
part  assurer  la  coherence  syntaxique,  lexicale  et 
semantique  de  l'expertise  formalisee.  L'expertise 
analysee  a  permis  de  degager  les  fonctionnalites  du 
"copilote  electronique"  par  rapport  aux  thematiques 
que  sont  la  gestion  de  l'aeronef,  le  suivi  de  la 
mission,  les  aspects  tactiques  sol-air  et  air-air. 

L'objet  de  l'etude  n'etait  pas  de  definir  avec 
precision  une  interface  mais  de  faire  des 
recommandations  a  2  niveaux  (22).  Le  premier 
niveau  est  celui  des  grands  principes  d'interaction 
avec  le  pilote.  C’est  ainsi  que  le  "Copilote 
electronique"  est  un  conseiller  qui  ne  prend  en 
aucune  fa9on  le  dessus  sur  le  pilote.  Le  pilote  peut 
ignorer  l'assistance,  voire  l'eteindre  s'il  la  trouve 
inutile.  La  pertinence  des  fonctionnalites  proposees 
depend  fortement  de  leur  contexte  d'application. 
Certaines  fonctionnalites  peuvent  etre  tres  utiles  au 
pilote,  mais  les  contraintes  de  la  tache,  comme  la 
pression  temporelle,  les  rendent  obsoletes  dans 
certaines  phases  de  vol.  L'aide  fonctionne  en 
permanence  de  fa9on  transparente  pour  le  pilote 
afm  d'evaluer  les  situations  sans  qu'il  n'y  ait  de 
dialogue  homme-machine.  Le  dialogue  n’est  initie 
que  lorsque  le  pilote  demande  une  evaluation  ou  un 
conseil,  ou  lorsque  l’assistance  s’ape^oit  que  les 
objectifs  de  securite  et  de  performance  ne  sont  plus 
respectes.  Pour  des  niveaux  d'assistance  eleves 
(replanification,  hypothese  "what-if’),  le  dialogue 
est  complexe  et  ne  peut  s'envisager  qu'avec  des 
moyens  de  dialogue  elabores  (commande  vocale, 
synthese  vocale,  etc.).  Le  second  niveau  de 
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recommandations  a  consiste  a  definir  sur  la  base 
d'un  aeronef  existant,  les  regies  precises  de  dialogue 
en  ce  qui  concerne  les  modes  d’interaction  homme- 
machine  ainsi  que  la  syntaxe  et  la  semantique  du 
dialogue. 

La  derniere  action  etait  de  tirer  les  enseignements 
des  methodes  utilisees  et  d'evaluer  les  differents 
concepts  developpes  dans  le  cadre  de  l'etude.  Une 
maquette  de  demonstration  sans  contrainte  "temps 
reel"  a  ete  realisee  a  la  fin  de  l'etude.  Constitue  d’un 
ensemble  de  5  stations  de  travail  Unix  reliees  a  un 
modele  avion  et  a  une  interface  simplifiee,  le 
demonstrateur  a  permis  de  valider  les 
fonctionnalites  du  copilote  electronique  sur  la  base 
de  scenarios  realistes  ou  survenaient  des 
evenements  imprevus.  Les  resultats  de  cette  action 
sont  presentes  dans  le  chapitre  suivant. 

5.  ENSEIGNEMENTS  DE  L'ETUDE 
’COPILOTE  ELECTRONIQUE” 

L'integration  precoce  des  facteurs  humains  dans  le 
processus  de  conception  d'une  assistance  et  le  cout 
que  represente  le  recueil  et  1’exploitation  de 
1'expertise  operationnelle  apparaissaient  de  prime 
abord  pour  les  partenaires  industriels  comme  une 
contrainte.  II  faut  reconnaitre  que  les  concepts  d'une 
assistance  ecologique  n'ont  ete  integres  par  les 
industriels  qu'apres  un  travail  important  sur 
1'expertise  operationnelle  pour  comprendre  les 
enjeux  cognitifs  des  pilotes  et  degager  les 
fonctionnalites  potentielles  d'une  assistance.  Mais 
une  fois  ce  travail  effectue,  il  est  rapidement  apparu 
que  le  contact  avec  le  monde  operationnel  a 
constitue  pour  les  equipes  d'intelligence  artificielle 
une  veritable  stimulation  en  termes  de 
fonctionnalites  proposees  et  de  recherches  de 
solutions  techniques.  En  ce  sens,  les  connaissances 
acquises  par  ces  equipes  tant  au  niveau  des  modeles 
du  fonctionnement  cognitif  des  pilotes  que  des 
methodes  facteurs  humains  constituent  un  plus 
incontestable  pour  le  developpement  de  systemes  a 
base  d'intelligence  artificielle  utilisant  1'expertise 
des  operateurs. 

Lors  de  la  presentation  du  demonstrateur  aux 
pilotes,  les  principales  critiques  ont  porte  sur  la 
partie  visible  de  la  maquette,  c'est-a-dire  l'interface. 
Bien  que  les  contraintes  de  temps  reel  ne  fissent  pas 
parties  de  la  validation,  les  temps  de  reaction  du 
demonstrateur  furent  eux  aussi  a  l'origine  de 
nombreuses  remarques.  Rapidement,  1'evaluation 
s'est  orientee  vers  une  evaluation  technique  aux 
depens  de  1'evaluation  fonctionnelle  souhaitee.  II  est 
alors  clairement  apparu  la  difficulty  de  dissocier 
dans  le  cadre  de  tels  developpements  exploratoires, 
les  concepts  des  technologies.  Cela  signifie  que 
pour  l'avenir,  1'evaluation  doit  etre  envisagee  avec 


encore  plus  d'attention.  Les  points  techniques 
"bloquants"  se  doivent  d'etre  identifies  et 
eventuellement  contournes  par  les  procedures  de 
simulation  s'ils  ne  peuvent  etre  resolus. 

Au  cours  de  1’evaluation,  les  fonctionnalites  des 
assistances  proposees  par  le  copilote  electronique 
ont  ete  validees  par  les  pilotes.  devaluation  a  aussi 
permis  de  valider  la  dynamique  et  la  faisabilite  des 
aides  proposees.  Des  reserves  ont  cependant  ete 
emises  sur  les  aides  necessitant  des  niveaux  de 
dialogue  eleves,  mais  il  faut  souligner  que  la 
faiblesse  de  l'interface  simulee  rendait  difficile 
toute  representation  d’un  produit  final.  Les 
avantages  d'une  assistance  "intelligente"  sont 
apparus  surtout  pour  les  situations  a  faible  delai 
temporel.  Il  a  ete  communement  admis  que  l'aide  ne 
pouvait  pas  s’imposer  au  pilote  et  qu'il  faudrait 
probablement  envisager  des  niveaux  differents 
d'initiation  du  dialogue  avec  le  pilote  suivant  les 
situations  rencontrees.  Les  raisonnements  de 
1'assistance  ainsi  que  les  connaissances  utilisees  ont 
ete  reconnues  par  les  pilotes,  validant  par  la  meme 
les  methodes  d’extraction  et  de  formalisation  de 
1’expertise.  Bien  que  les  avantages  d'une  telle  aide 
apparaissent  comme  certains,  il  est  en  l'etat  actuel 
impossible  de  les  quantifier  car  : 

-  L'usage  de  l'aide  genere  forcement  une 
charge  de  travail  supplemental  pour  le  pilote. 
Cela  signifie  que  1'evaluation  de  l'aide  ne  doit  pas 
etre  centree  exclusivement  sur  sa  seule  utilisation 
mais  qu’elle  doit  integrer  la  situation  globale  de 
travail  et  les  resultats  attendus  sur  la  performance, 
la  securite  et  surtout  le  confort  "cognitif  du  pilote. 

-  La  globalite  d'une  aide  comme  le  copilote 
electronique  va  modifier  l'activite  du  pilote  et  ses 
relations  avec  les  systemes  embarques.  Il  est  done 
difficile  de  comparer  les  benefices  d'une  telle  aide 
avec  des  systemes  existants.  Une  evaluation 
operationnelle  ne  peut  s'envisager  que  par  rapport  a 
des  objectifs  qualitatifs  de  performance  et  de 
securite  independamment  des  moyens  pour  y 
arriver. 

-  Le  copilote  electronique  est  une  aide 
dynamique  et  adaptative  tout  au  long  de  la  mission. 
Son  evaluation  finale  ne  peut  se  faire  que  par  une 
mise  en  situation  la  plus  proche  possible  d'un 
contexte  reel  de  mission. 

L'utilisation  de  connaissances  operationnelles  dans 
des  systemes  mettant  directement  en  jeu,  la 
performance  et  la  securite  des  equipages  pose  le 
probleme  de  la  validite  de  1’expertise.  En  effet,  la 
validite  de  1’expertise  est  limitee  dans  le  temps.  Les 
systemes  de  navigation  et  d'armement  ainsi  que  les 
capteurs  evoluent,  les  tactiques  evoluent  en 
fonction  de  revolution  des  materiels  adverses,  les 
cadres  tactiques  changent  d'un  terrain  operationnel 
a  l’autre,  les  solutions  d’aujourd’hui  ne  seront  pas 
forcement  celles  de  demain.  Cette  instability  de 
1’expertise  constitue  un  des  points  les  plus  delicats  a 
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gerer.  L'expertise  est  un  tout.  Le  changement  cTurie 
connaissance  peut  avoir  des  repercussions  sur 
Tensemble  de  l’expertise.  Les  evolutions  sont 
cependant  des  evolutions  de  "contenu"  et  non  de 
"contenant".  Les  mecaniques  de  traitement  de 
l'information  et  de  gestion  des  compromis  sont 
stables  et  ne  remettent  pas  en  cause  I’architecture 
d’un  systeme  d’aide.  Cependant  il  est  clair  qu’une 
assistance  "intelligente"  devra  etre  mise  a  jour 
regulierement  ou  alors  etre  dotee  de  capacites 
d’apprentissage.  Poursuivre  dans  cette  voie,  c’est 
envisager  des  aujourd’hui,  des  recherches  pour 
developper  des  systemes  automatiques 
d’enrichissement  et  de  validation  de  la  base  de 
connaissances. 

Une  question  sous-jacente  a  la  realisation  d’un 
systeme  potentiellement  embarquable  est  que  Ton 
ne  peut  ignorer  sa  certification.  Les  regies  actuelles 
ne  sont  pas  adaptees  aux  technologies  mettant  en 
jeu  des  systemes  a  base  de  connaissances 
operationnelles.  Au-dela  de  la  validite  de  la  base  de 
connaissances,  il  faut  certifier  les  outils 
d’intelligence  artificielle  ainsi  que  leur  utilisation 
par  les  personnels  navigants.  Comment  certifier  un 
systeme  dont  I’optimalite  n’est  pas  de  trouver  la 
meilleure  solution  mais  d’aider  le  pilote  a  trouver 
une  solution  qu’il  saura  realiser  et  qui  lui  permettra 
de  remplir  sa  mission  en  toute  securite  ?  Ces 
questions  sont  ouvertes  et  devront  etre  abordees 
dans  un  futur  proche. 

5.  CONCLUSION 

Le  programme  copilote  electronique  est  une 
premiere  tentative  pour  apporter  une  aide 
cooperative  a  des  pilotes  d’avion  d’armes.  La 
necessite  de  laisser  le  pilote  dans  la  ’’boucle"  et  de 
1’assister  afin  d’optimiser  ses  savoirs  et  savoir-faire 
tout  en  lui  laissant  une  totale  gestion  de  la  situation 
constituent  les  points  forts  de  l’aide  proposee. 
Realise  sur  3  ans,  le  programme  a  eu  comme 
originalite  d'integrer  des  les  phases  initiales  des 
recommandations  facteurs  humains  provenant  de 
1’analyse  de  1'activite  des  pilotes  de  combat. 
Associant  les  techniques  de  Lintel  ligence 
artificielle,  le  copilote  electronique  est  un  systeme  a 
base  de  connaissances  expertes.  Les  methodes 
developpees  pour  elaborer  la  base  de  connaissances 
et  definir  les  fonctionnalites  de  l’aide  ont  ete 
validees.  Le  demonstrates  presente  a  Tissue  de 
1’etude  permet  de  tirer  les  premiers  enseignements. 
Le  programme  a  permis  de  valider  les  assistances 
proposees  ainsi  que  leurs  faisabilites.  Si  les 
concepts  ont  pu  etre  valides,  il  est  certain  que  les 
benefices  d’une  telle  aide  sont  encore  difficiles  a 
evaluer.  Cela  ne  pourra  etre  envisage  que  dans  des 
developpements  ulterieurs  si  Ton  prend  soin  de 
lever  les  contraintes  d'interface  et  de  temps  reel. 


Malgre  ces  limitations,  un  programme  comme  celui 
du  copilote  electronique  a  permis  de  developper  un 
savoir-faire  "facteurs  humains"  et  industriel  qui 
peut  etre  utilise  pour  des  applications  moins 
ambitieuses  a  base  de  connaissances  expertes.  Il  a 
surtout  permis,  meme  si  certaines  questions  ne  sont 
pas  encore  resolues,  d’ouvrir  la  porte  vers  de 
nouveaux  types  d’aides  qui  seront  peut-etre  les 
seules  solutions  pour  vraiment  laisser  a  Toperateur 
humain  Tentiere  responsabilite  des  decisions  prises 
dans  les  futurs  systemes  complexes. 
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The  challenge:  More  information,  still  for  humans 

Information  is  a  key  element  for  success  or  failure 
on  future  battlefields.  Continuous  advances  in 
information  technology  and  battle  management 
systems,  especially  growing  computer  capacity  and 
interoperability  promise  to  provide  comprehensive 
tactical  situation  awareness  down  to  unit  level, 
thereby  improving  mobility,  survivability  and 
sustainability  of  today’s  weapon  systems. 

However  increased  availability  of  information  in 
the  computerized  support  systems  does  not 
automatically  lead  to  increased  usability.  It  rather 
may  lead  to  information  proliferation,  hidden 
information  and  pertinent  problems  regarding 
operator  information  processing.  These  problems 
even  grow  under  time  pressure  in  a  stressful 
environment.  Are  these  problems  unavoidable?  Or 
is  there  a  solution  to  handle  the  overwhelming 
amount  of  information  which  tomorrows  battle 
management  systems  and  personal  have  to  work 
on? 

In  aviation  there  were  tremendous  technological 
efforts  during  the  last  twenty  years  to  answer 
similar  questions  through  increase  of  automation 
like  the  introduction  of  flight  management  systems 
or  fully  computerized  „glass  cockpits4*.  Again,  to 
the  surprise  of  many  people,  the  relative  safety  did 
not  increase,  but  remained  almost  constant  [Billings 
1997]. 

The  upcoming  solution:  Cognitive  Automation  and 
Assistant  Systems 

These  problems  led  to  discussions  and  doubts  about 
the  benefit  of  automation  on  the  one  hand,  and 
research  in  favor  of  “cognitive  automation”  on  the 
other  hand.  As  opposed  to  increased  conventional 
automation  in  the  sense  as  mentioned  above, 
cognitive  automation  is  based  on  cognitive 
engineering  (e.g.  [Rasmussen  et.al.  1994])  and 
more  adapted  to  interact  with  human  cognition 
[Onken  1998],  This  gives  the  chance  to  handle 
more  information  in  the  cockpit  without  decreasing 
usability. 

Prototypes  of  cognitive  automation  in  aviation  are 
the  Cockpit  Assistant  SYstem  CASSY  for  civil 
IFR,  flight  tested  in  1994,  and  CAMA,  the  Crew 
Assistant  Military  Aircraft,  developed  together  with 
DASA,  DLR  ,  ESG  and  the  University  of  Armed 


Forces.  Simulator  trials  were  conducted  in  1998, 
flight  tests  are  scheduled  for  2000,  e.g.  [Lenz  & 
Onken  2000]  in  this  proceeding. 

But:  How  can  we  be  sure  that  no  new  problems  will 
arise  with  cognitive  automation? 

Undoubted,  conventional  automation  was 
motivated  by  positive  intentions.  One  major  intent 
was  the  reduction  of  workload.  The  effect  was  so 
enormous  that,  as  a  result,  we  face  now  a  “pilot- 
out-of  -the-loop”  problem,  e.g.  [Endsley  &Kiris 
1995],  the  “ironies  of  automation”  [Bainbridge 
1987]  and  operators  speaking  of  “99%  boring,  1% 
panic”  [Kraiss  1994]. 

How  can  we  be  sure  that  cognitive  automation 
solves  problems  but  does  not  raise  new  problems? 
If  we  can  not  be  sure,  how  can  we  learn  from  the 
lessons  and  implement  ergonomics  /  human  factors 
right  from  the  start  of  the  development  cycle? 

Ergonomics  /  human  factors  offer  a  wide  range  of 
methods  for  detection  and  handling  of  usability 
problems.  On  the  other  hand,  even  well  experienced 
concepts  like  e.g.  workload  more  often  fail  to 
reliably  describe  the  problems,  especially  with 
increasing  technical  complexity  or  „self  animated44 
machines  [Sarter  &  Woods  1994].  How  can  we 
implement  newer  concepts  like  usability  [Nielsen 
1993]  or  situation  awareness  [Endsley  1995],  how 
can  we  detect  problems  like  cognitive  fixation  or 
dangerous  attention  distribution? 

How  can  we  meet  the  often  different  demands  of 
our  target  groups  such  as  engineers,  managers, 
scientists  and  operators? 

How  can  we  bridge  the  gap  between  the  diametrical 
poles  “subjective  /  objective”,  “intuitive  / 
analytical”,  “global  /  detailed”  or  “scientifically 
exact  /  efficient”  in  order  not  only  to  detect  but  to 
solve  usability  problems? 


Paper  presented  at  the  RTO  HFM  Symposium  on  u  Usability  of  Information  in  Battle  Management 
Operations  ”,  held  in  Oslo ,  Norway ,  10-13  April  2000,  and  published  in  RTO  MP-57. 
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A  prototype  for  integrated  usability  testing: 
caSBAro 


As  an  answer  to  these  needs  a  new  kind  of  usability 
testing  tool,  caSBAro,  was  developed  in  parallel 
with  CAMA.  The  acronym  stands  for: 


c  omputer 
a  ided 

S  ituation  and 
B  ehavior 
A  nalysis 
r  eplay  and 

o  nline 


supports  not  replaces  human 
factors  analysis 
analysis  of  behavior  cannot 
be  done  without  analyzing  the 
underlying  situation 
the  record  can  be  fully 
replayed  in  a  flight  simulator 
all  caSBAro  analysis  modules 
must  be  capable  to  work  in 
realtime  for  the  future  option 
to  plug  them  into  the  assistant 
system 


Figure  1  shows  the  structure  of  caSBAro:  a  generic 
flight  simulator,  eye-  and  headtracker,  digital 
videodisc  system  and  recording  /  visualization  / 
analysis  of  man-  and  machine  behavior. 

One  core  element  of  caSBAro  is  the  sharpening  of 
our  best  usability  measuring  tools,  our  pilots,  by 
offering  them  a  full  mission  replay  in  the  simulator 
including  the  eye  tracking  records.  This  gives 
engineers,  managers  and  operators  the  platform  for 
a  very  detailed  debriefing  without  memory  effects, 
an  intuitive  access  to  objective  data  analyzable 
down  to  the  byte  and  eyeblink  level  [Flemisch  & 
Onken  1999]. 

Another  core  element  of  caSBAro  and  focus  of  this 
paper  is  the  analysis  of  the  operators  interaction 
resources,  especially  the  distribution  of  the  visual 
resource  in  the  cockpit.  This  gives  an  almost  direct 
access  to  the  visual  part  of  the  human  bottleneck 
and  usability  problems  like  information  overload  or 
dangerous  attention  distribution. 


Workstation(s) 

Vision 


Cognitive 

Assistant 

System 


AS-DM 


Flightsim 


Object 
detection  { 


analysis 


Harddisc 


Scene  video 
(point  of  gaze) 


Situation  Visualisation  Behavior  Visualisation 


head  data 


Raw  eye 
video 


Figure  1 :  caSBAro 
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Experimental  series  on  “variation  of  technical 
support  for  manual  flying  and  navigation" 


C-160  ‘Tran sail”.  The  experiments  were  embedded 
into  a  2  days  /  pilot  simulation  campaign. 


The  main  objectives  for  the  following  series  of 
generic  simulator  experiments  (or  better:  quasi 
experiments  in  the  rigorous  sense  of  classical 
experimental  psychology)  were: 

•  estimation  of  the  method’s  overall  sensivity  for 
the  visual  resource, 

•  estimation  of  the  method’s  potential  in  the 
ergonomical  toolbox,  compared  to  the  classical 
methods  “subjective  workload”  and  “objective 
performance”, 

•  exploration  of  relationship  between  different 
technical  supports  and  their  effects  on  the 
operator’s  visual  resource  in  order  to  improve 
the  assistant  system  CAM  A. 

The  subjects  were  6  military  pilots  of  a  German 
Tactical  Air  Transport  Wing  (LTG61  Landsberg), 
aged  30  -41  (average34)  with  a  experience  of  800  - 
6000  (average  2700)  flight  hours  on  several  aircraft 
types,  especially  the  two  engine  transport  aircraft 


The  task  performed  by  the  pilots  consisted  of  a 
combination  of  two  subtasks,  a  tracking  subtask 
with  higher  frequency  (manual  low  level  flight  of  a 
preplanned  minimum  risk  route),  and  a  low 
frequency  supervision  /  navigation  subtask.  Each 
subtask  was  supported  by  different  technical  means. 

On  the  one  hand  this  prototype  combination  of 
subtasks  is  quite  relevant  for  the  aviation  domain, 
on  the  other  hand  it  promised  to  be  prototypical 
enough  to  allow  a  transfer  of  experience  into  other 
domains. 

The  scenario  consisted  of  a  preplanned  low  level 
minimum  risk  route  with  about  7min  flight  time  in 
a  hilly  area  (Black  Forest),  a  dynamic  threat  theater 
with  simulated  hostile  SAM-stations  (Surface-to- 
Air  Missiles)  and  an  ACO  (Airspace  Control  Order) 
with  egress  corridors. 


ary  display 
tical  situation 


ADI  or  3D-display 

WM  ;i 

navigation  display 


head  mounted  eye-  and  head  tracker 


outside  vision 


Figure  2:  simulator  with  displays  and  eye  tracking  equipement 
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The  subtask  F  “Manual  Flight”  demanded  flying  of 
the  minimum  risk  route,  which  remained  constant 
through  all  experiments,  under  VFR-conditions 
(Visual  Flight  Rules)  and  “as  accurately,  fast  and, 
most  important,  safely  as  possible”.  The  technical 
support  for  this  subtask  was  varied  as  follows: 

ADI:  Classical  combination  of  cockpit  instruments 
with  artificial  horizon,  speed,  altitude,  radar  altitude 
etc.  as  in  state  of  the  art  civil  aircraft  glass  cockpit 
aircraft  (as  shown  in  figure  2). 

3D:  newly  developed  flight  guidance  display  with 
three-dimensional  dynamic  picture  of  terrain 
elevation,  terrain  features  and  a  “tunnel  in  the  sky” 
of  the  minimum  risk  route  (as  shown  in  figure  2), 
(by  ESG,  see  also  [Schulte  &  Stutz  1998]). 

3DADI:  newly  developed  combination  of  ADI  and 
3D,  much  smaller  3D-display  area. 

Auto:  no  manual  but  automated  following  of  the 
minimum  risk  route,  [Bamberger  &  Lenz  1998]. 

The  subtask  N  “Navigation”  consisted  of 

1.  Monitoring  of  the  tactical  situation  on  the 
secondary  display  with  regard  to  changes. 

2.  In  case  of  changes:  decision  whether  own  route 
or  egress  corridor  is  endangered,  callout 
(“threat  factor  /  no  factor”). 

3.  If  route  is  endangered:  choice  of  alternative 
egress  corridor,  callout  of  choice,  and 

4.  finally  replanning  by  selecting  the  alternative 
corridor  on  the  Secondary  display 
(touchscreen),  then  selecting  button  “Replan 
via”  on  the  Navigational  Display. 

After  that  the  replanning  sequence  was  terminated, 
the  original  minimum  risk  route  remained  constant 
during  all  experiments. 

The  technical  support  for  this  subtask  was  varied  as 
follows: 

No  support:  “only”  visualization  of  the  tactical 

situation  on  the  Secondary  Display. 

Highlighting:  highlighting  of  changes  by 

different  color  and  blinking  symbols. 


Callout:  (in  addition  to  highlighting)  a 

speech  output  “tactical  situation  changed”. 

Proposal:  (in  addition  to  highlighting  and 

callout)  a  machine  generated  solution  by  speech 
output,  .e.g.  “replan  via  corridor  TK05”, 
highlighting  of  the  alternate  corridor  and  textual 
feedback  on  the  navigation  display. 

Simplified  activation:  (in  addition  to  all  support 
mentioned  above)  the  simple  activation  of  a 
proposal  by  selecting  a  “Roger  Do  It”  button  or 
alternatively  by  a  speech  input  “roger  do  it”. 

Variation  and  combination  of  subtasks  and  support: 

Comparison  I  (E  l  -  E_4)  investigates  subtask  F 
“manual  flight”  with  different  technical  support,  but 
with  no  navigation  (See  also  table  1). 

Comparison  II  (E_5  -  E_9)  addresses  the 
navigational  subtask  N  with  different  technical 
support,  combined  with  a  pseudo  flight  task 
“supervision  of  automated  low  level  flight”.  These 
conditions  are  comparable  to  those  of  a  PNF  (Pilot 
Non  Flying)  busy  with  a  navigational  task. 

Comparison  III  (E_15,  E_12,  E_14,  E_ll)  deals 
with  the  combinations  of  the  two  subtasks  with 
none  or  complete  support.  The  idea  was  that 
extreme  combination  of  support  would  also 
generate  extreme  behavior  and  would  therefore 
stretch  out  the  behavioral  spectrum  in  a  manner  that 
in  between,  nonextreme  combinations  can  be 
derived  at  least  qualitatively  by  interpolation  of 
extreme  combinations  without  being  measured 
explicitly.  The  simple  but  striking  reason  behind 
this  is  the  limited  maximum  time  for  the 
experiments  due  to  the  weight  and  pertinent 
discomfort  of  the  head  mounted  equipment. 

To  minimize  effects  of  order  of  the  test  runs,  they 
were  not  conducted  in  logical  order,  but  were,  after 
placing  the  order-critical  experiments  (see  chapter 
collision  aircraft),  varied  according  to  a  replicated 
Latin  square  design  (see  also  [Johannsen  &  Rouse 
1983]). 


subtask  F  ^manual  flight44 

autopilot 

flight  guidance  display 

ADI  3D 

no  Nav. 

Comparison  I  -> 

E_4 

E_1  E_2 

E_3 

display  only 

E  5 

E_15  E  12 

+  highlighting 

E_6 

a 

z,  -S 

■*  6JD 

+  callout 

E  7 

Comparison  III 

t: 

+  proposal 

E  8 

.Zr 
«  > 

2  l 

S  * 

w 

CL 

CL 

3 

c/J 

+  simplified  activation 

E_9 

E_ll  E_14 

Comparison  II 

Table  1:  Variation  of  technical  support,  combination  of  subtasks 
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Eye  tracking  data 

Figures  3-5  represent  the  distribution  of  the  visual 
resource  across  the  visual  workspace  for  the 
specific  subtask  /  support  combination,  averaged 
over  all  pilots  and  flighttime.  The  lighter  the  areas 
are,  the  more  fixation  time  (in  this  case 
corresponding  to  visual  attention)  pilots  spent  on 
that  particular  spot  (excluding  warm  up  phase, 
exponentially  accumulated  fixation  time,  shifted  to 
positive  values,  standardized  to  volume  integral  and 
projected  into  2D,  graphical  representation  by 
caSBAro-XRT,  [Morawski  1999]). 

The  white  %-numbers  represent  the  average 
percentage  of  visual  attention  on  the  specific  region 
of  interest  (displays,  outside  vision) 

Subjective  workload  with  SWAT  rating 

In  order  to  allow  comparison  of  eye  tracking  with 
classical  approaches,  the  subjective  mental 
workload  of  the  pilots  was  measured  with  the 
SWAT  method  (Subjective  Workload  Asessment 
Technique).  According  to  this  method,  mental 
workload  contains  three  components,  time  pressure 
T,  mental  effort  E  and  stress  S  in  three  stages,  low 
1 ,  medium  2  and  high  3. 

The  TES-triple  in  figures  3  -5  represent  the  pilots’ 
median  postflight  estimation  of  subjective 
workload. 

W  represents  the  mean  value  of  the  conjoint 
subjective  workload.  This  “conjoint  scaling” 
method  also  takes  into  account  interpersonal 
differences  in  the  relative  importance  of  T,  E  and  S. 
Part  of  this  method  is  that  pilots  sort  the  27  possible 
SWAT-combinations  in  order  of  relevance  before 
the  experiments  [Nygren  1991]. 

Performance  PF  for  subtask  F  “Manual  Flight” 


As  the  above  mentioned  subjective  workload  is 
only  sensitive  for  the  overall  task  combination,  the 
relationship  between  technical  support  and  specific 
subtask  must  be  evaluated  by  subtask  sensitive 
methods.  Subjective  methods,  e.g.  Cooper-Harper- 
Scale,  would  also  be  usable  here,  but  because  of  the 
caSBAro  capability  for  recording  aircraft 
parameters,  the  calculation  of  a  “mean  distance  to  a 
specified  track”  dm  as  most  frequently  used  method 
for  objective  performance  assessment  can  easily  be 
done. 

Mean  speed  iasm  helps  to  detect  potential  speed 
accuracy  tradeoffs. 

Performance  PN  for  subtask  N  “Navigation” 

Like  for  PFi  speed  accuracy  tradeoffs  also  have  to  be 
controlled  for  PN.  This  is  done  by  two  values 
representing  time  and  accuracy:  overall  time  for 
solving  a  conflict  and  percentage  of  correct  / 
successful  reaction. 


Moreover  this  subtask  can  be  structured  with 
respect  to  the  different  stages  of  human  information 
processing,  e.g.  according  to  [Wickens  1992]: 

1.  perception,  here  detection  (and  callout)  of  a 
potential  conflict  (step  1  and  2  of  the 
description  for  subtask  N  above). 

2.  decision  and  response  selection,  here  selection 
of  a  alternative  egress  corridor  (and  callout). 

3.  response  execution,  here  activation  of  a 
replanning  process. 

Because  a  specific  technical  support  can  have 
different  effects  on  different  stages,  average  time 
and  quality  percentage  was  calculated  for  each 
specific  stage.  In  order  to  highlight  the  overall 
effect,  only  correct  reaction  were  accumulated  over 


the  three  stages.  Table  2 
referring  to  figure  4  E_5: 

provides  an 

example 

Single  stage 
performance 

Accumulated 

overall 

performance 

time 

perception 

82% 

3.7s 

selection 

88%  +  2.0s 

72% 

5.7s 

execution 

60%  +  3.6s 

43% 

9.3s 

Table  2:  objective  performance  of  subtask  N, 
example  from  figure  4,  E_5 

This  means  that  within  this  subtask/support 
combination,  averaged  over  6  pilots,  82%  of  all 
navigational  conflicts  (changes  of  tactical  situation 
that  endangered  the  preplanned  route)  were 
detected  and  called  out  by  the  pilots  after  3.7 
seconds.  2  seconds  later  88%  of  these  conflicts 
were  also  solved  (and  the  solution  called  out) 
correctly,  3.6  seconds  later  60%  of  these  solutions 
were  also  executed  correctly,  so  that  43  %  of  all 
conflicts  were  solved  correctly  after  9.3  seconds, 
57%  were  incompletely  replaced  by  a  subsequent 
conflict  or  failed  at  one  or  the  other  stage  of  the 
pilot’s  information  processing. 
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Comparison  I:  Variation  of  subtask  F  “Manual 
Flight”,  no  Navigation 

Comparison  I  looks  at  the  isolated  subtask  F 
“Manual  Flight”  with  different  flight  guidance 
support  ADI,  3D,  3DADI  and  automatic  flight.  In 
figure  3  e.g.  “PF+”  stands  for  an  improvement  in 
flight  performance,  “W  -  ’  for  an  almost  constant 
subjective  workload.  Black  arrows  show  a  virtual 
flow  of  visual  attention  between  two 
configurations. 

E  1  ADI  represents  the  classical  low  level  flight 
under  VFR  conditions  (Visual  Flight  Rules)  and 
with  state  of  the  art  displays:  Subjective  conjoint 
workload  W  is  average  with  42%.  This  subtask  and 
configuration  is  the  daily  but  nevertheless  not  easy 
job  of  these  pilots.  Visual  attention  is  mostly  (56%) 
directed  to  the  outside  vision,  where  e.g.  hill  ridges 
are  fixated  in  order  to  avoid  terrain  collisions.  The 
visual  scanning  pattern  of  the  ADI  is  characterized 
by  a  classical  “basic  T”,  a  repetitive  change 
between  speed,  artificial  horizon  and  altitude  /  radar 
altitude  /  variometer.  Short  gazes  downwards  to  the 
Navigational  Display  are  used  to  detect  deviations 
from  the  minimum  risk  route  and  to  perform 
medium-term  orientation  (“ok,  after  the  next  ridge 
right  into  the  valley,  then  one  mile  straight  on, 
uups...”). 

E  2  3D  is  the  same  flight  with  3D-display:  Visual 
attention  is  attracted  by  the  integrated  information 
of  terrain,  aircraft  attitude  and  minimum  risk  route 
on  the  3D-display.  This  limited  visual  resource  is 
withdrawn  mostly  from  the  outside  vision  and 
partly  from  the  navigational  display.  Some  pilots 
urge  themselves  to  check  the  outside  environment 
more  frequently  (max.  35%),  others  just  abandon 
this  source  of  information  (min.  4%).  Flight  path 
accuracy  as  measurement  of  objective  performance 
is  almost  4  times  higher  than  with  classical  ADI, 
speed  is  higher,  subjective  workload  is  clearly 
reduced. 


E  3  3DADI  is  the  hybrid  of  classical  ADI  and  3D: 
The  concentration  effect  already  observed  in  E_2 
even  grows  stronger,  performance  is  almost  equal, 
subjective  workload  is  increased  due  to  the  small 
size  of  the  3D- window,  but  is  still  lower  than  El. 

E  4  autopilot  with  pilot  as  supervisor:  Even  though 
the  autopilot  configuration  is  quite  convenient 
(lower  flight  path  accuracy  and  speed  as  flown  by 
the  pilots  themselves),  subjective  workload  is 
higher  than  in  e.g.  E_2.  When  asked  about  these 
surprising  ratings  pilots  stated  a  “natural  distrust” 
of  automated  flight  due  to  lack  of  experience  and 
short  reaction  time  in  case  of  malfunction. 

The  automation  frees  visual  resources,  which  flow 
into  the  secondary  and  the  navigational  display, 
nevertheless  the  overall  distribution  of  visual 
attention  is  quite  similar  to  E_2.  As  e.g.  the 
scanpath  theory  [Stark  &  Choi  1996]  formulates  a 
strong  relationship  between  observed  visual 
behavior  and  internal  mental  representation  of  a 
visual  task,  we  can  therefore  assume  that  the  visual 
parts  of  “flying  an  aircraft”  and  “supervising  a 
machine  flying  an  aircraft  human-like  ”  have  quite 
similar  mental  representations.  This  affirms  e.g. 
efforts  like  [Schulte  1996],  who  investigated  visual 
behavior  of  pilots  in  low  level  flight  by  stimulating 
them  with  a  movielike  video  replay  of  a  real  flight 
in  a  simulator  with  outside  view. 
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E_1 

W  42% 

T  1  E  2  S  2 

E_2 

F:  ADI 

dm0.118nm 

iasm227  knots 

F:  3D 

N:  - 

N:  - 

W  9%  T  1  E  1  S  1 

dm  0.028  iasm255  knots 


E_3 

W  21% 

T  1  E  2  S  2 

E_4 

W  22% 

T  1  E2  S  1 

F:  3DADI 

N:  - 

dm  0.032 

iasm253  knots 

F:  AutO 

N:  - 

dm  0.036 

iasm205  knots 

Figure  3:  Comparison  I,  flight  with  different  displays,  no  navigation 
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Comparison  II:  Variation  of  Navigation,  Autopilot 

Comparison  II  (figure  4)  investigates  the  influence 
of  different  technical  support  for  subtask  N 
“Navigation”  without  subtask  F. 

E  5  without  support:  89%  of  the  visual  attention  is 
located  at  the  secondary  display,  only  sometimes 
gazes  are  moving  elsewhere,  e.g.  to  the  outside 
view.  Subjective  workload  is  lower  than  e.g.  E_4 
(supervision  of  autopilot),  objective  performance  is 
only  medium  mainly  because  of  execution 
interruptions  by  new  conflicts.  This  gives  evidence 
that  the  experiment  is  working  close  to  the  upper 
limits  of  performance  and  is  therefor  sensitive. 

E  6  with  highlighting:  rate  and  speed  of  detection 
increases.  Reasons  for  that  might  be  a  better 
detectability  in  peripheral  vision  and  a  faster 
discrimination  between  endangering  and  harmless 
tactical  elements.  The  values  for  selection  and 
response  execution  together  with  pilots’  comments 
could  be  a  hint  that  the  improvement  is  partially 
compensated  by  distracting  effects  caused  by  the 
symbol  blinking. 

E  7  with  additional  speech  output  in  case  of  a 
tactical  change:  performance  and  subjective 
workload  are  almost  unchanged  compared  to  E_6, 
but  a  fundamental  quantitative  and  qualitative 
change  of  the  visual  behavior  can  be  observed:  Free 
visual  resources  almost  doubled.  The  attentional 
field,  which  was  almost  exclusively  focused  to  the 
navigational  task  /  secondary  display,  is  partially 
freed  now.  In  contrast  to  E_6,  the  complete  right 
side  of  outside  vision  can  be  covered  now. 

E  8  with  additional  proposal  for  conflict  resolution: 
high  improvement  of  response  selection,  slight 
reflux  of  visual  attention  into  the  secondary  display. 
However,  regarding  the  overall  performance  an 
almost  paradox  effect  can  be  observed:  Although 
pilots  know  the  conflict  solution  much  faster  than 
the  machine,  they  tend  to  wait  for  the  proposal  to 
assure  themselves.  So  they  loose  precious  time  for 
the  execution  before  the  next  conflict  occurs.  This 
effect  could  of  course  also  happen  in  reality,  but  the 
observed  effects  on  the  overall  performance  can  be 
considered  as  an  artifact  caused  by  the  experimental 
conditions,  especially  the  relative  simplicity  of  the 
navigational  task. 

E  9  with  simplified  activation  by  “roger  do  it” 
button  or  speech  input:  The  “waiting  for  the 
proposal”  effect  is  still  observable,  but  these 
proposals  are  activated  fast  and  accurate,  so  that 
compared  to  unsupported  E__5  overall  time  is  equal, 
but  quality  doubles!  Freed  visual  resources  can 
flow  in  other  information  sources. 


Comparison  III:  Extreme  combination  for  flying 
and  navigation 

Comparison  III  (figure  5)  investigates  the  extreme 
combinations,  ADI  or  3D  for  manual  flying 
subtask,  no  support  or  full  support  including 
proposal  and  simplified  activation  for  the 
navigation  subtask. 

E  15  -  flying  with  ADI,  navigation  with  no  support 
is  -  not  surprising  -  the  experiment  with  the  highest 
subjective  workload.  The  flying  subtask  is, 
compared  to  E_1  with  no  navigation,  performed 
without  major  dropouts,  even  with  20%  of  the 
visual  resource  withdrawn  from  this  subtask  and 
used  for  the  navigational  subtask.  Obviously  this  is 
not  enough  to  perform  this  subtask  sufficiently, 
leads  to  the  lowest  success  of  12%  and  a  SWAT 
stress  value  of  3  for  all  pilots.  Remarkable  is  the 
still  successful  rule  of  prioritization  “aviate  - 
navigate  -  communicate  -  manage  systems” 

A  closer  look  at  the  extreme  transfer  from  E  15  to 
full  support  E  14  (diagonal  arrow  in  the  center  of 
figure  5)  shows  a  dramatic  reduction  of  subjective 
workload  and  a  huge  improvement  of  the  sub  tasks’ 
performance,  especially  for  the  navigation  subtask. 
Regarding  the  visual  resources,  the  percentage  of 
the  three  information  sources  navigation  display, 
secondary  display  and  outside  vision  is  reduced  to  a 
half  and  focused  to  the  3D  display  (triplication). 
The  detailed  mechanism  of  this  resource  flow 
becomes  transparent  by  a  closer  look  to  the 
intermediate  combinations: 

The  transfer  from  E  15  to  E  12,  ADI  to  3D  with 
unsupported  navigation,  leads  to  an  improvement  of 
flight  performance  with  a  concentration  of  visual 
resources,  flowing  from  outside  vision  and 
navigation  display  into  the  3D  display,  an  effect 
that  can  also  be  seen  in  Comparison  I.  Better 
support  for  the  subtask  F  does  not  only  improve 
flying,  moreover  freed  resources  can  be  used  for  the 
navigation  subtask,  visible  in  a  higher  percentage  of 
the  visual  resource  allocation  in  the  secondary 
display  and  a  better  performance  on  all  stages  of 
information  processing. 

Adding  the  navigation  support  (E  l  4)  now  leads  to 
an  acceptable  performance  of  navigation  with 
almost  constant  flight  path  accuracy. 
Simultaneously  freed  visual  resources  can  be 
reinvested  into  the  outside  vision. 

A  similar  picture  can  be  developed  by  following  the 
circle  counterclockwise  from  E  l 5  via  E  ll  to 
E  l 4:  additional  navigation  support  in  E  ll  leads 
to  a  higher  navigation  performance,  which  reaches 
not  yet  the  maximum  of  E  l 4. 
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E  5 


W11%  T 1  E  1.5  S 1  E  6 


W  5%  T  1  E  1  S  1 


F:  Auto  per  82%  3.7s 

N:  no  support  sel  88%+2.0s  72%  5.7s 

exe  60%+3.6s  43%  9.3s 


F:  Auto  per  92%  1 .9s 

N:  highlighting  sel  83%+2.8s  76%  4.7s 

exe  66%+4.9s  50%  9.6s 


E_7  (right  side)  W4%  T1  El  Si 

F:  Auto 

N:  highlight.  +  callout 

per  93%  2.0s 

sel  88%+2.6s  82%  4.6s 
exe  66%+5.1s  54%  9.7s 


W0% 


W  4% 


F:  Auto 

N:  highlight.+callout 
proposal 

simplified  activation 


F:  Auto 

per  94%2.2s  N:  highlight.+calloutper  92%  2.2s 

sel  97%+6.0s  91%  8.2s  proposal  sel  98%+5.7s  90%  7.9s 

exe  100%+1 .2s  91%  9.4s  exe  31%+4.9s  28%  12.8s 


Figure  4:  Comparison  II,  automatic  flight  with  navigational  support 
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Simultaneously  freed  resources  flow  back  to  the 
subtask  F.  These  resources  are  reinvested  not  so 
much  into  the  outside  vision  -  obviously  this 
percentage  is  already  high  enough  compared  to  e.g. 
E  12  -  but  more  into  the  ADI. 


The  transfer  from  E  ll  to  E  l 4  once  again  shows 
the  effects  of  the  3D  display,  improvement  of  flight 
quality  and  concentration  of  visual  resources. 


E_15 

W  82% 

T  2.5  E  3  S3 

E_12 

W  51% 

T  2  E  2  S  2.5 

F:  ADI 

dm  0.106nm 

iasm245  knots 

F:  3D 

dm  0.026 

iasm234  knots 

N:  no  support 

per 

68%  5.8s 

N:  no  support 

per 

79%  3.2s 

sel  56%+3.6s 

38%  9.4s 

sel  67%+3.9s 

53%  7.1s 

exe  31%+4.9s 

12%  14.3s 

exe  38%+4.7s 

20%  11.8s 

F:  ADI  dm  0. 1 34  iasm239  knots 

N:  highlight.+callout  per  82%  4.0s 

proposal  sel  91%+5.5s  75%  9.5s 


F:  3D  dm  0.030  iasm224  knots 

N:  highlight.+callout  per  89%  3.7s 

proposal  sel  91%+5.1s  81%  8.8s 

simpl.activation 


Figure  5:  Comparison  III,  extreme  combinations 
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Test  with  collision  aircraft 

The  observed  concentration  of  visual  resources  into 
the  3D  display  is  not  negative  by  itself,  the 
performance  improvements  are  quite  impressing. 
Nevertheless  pilots  and  evaluators  had  uneasy 
feelings  after  a  look  at  the  eye  tracking  videos  and 
resource  distribution.  The  reason  is  the  -  until  now 
unmentioned  nevertheless  vital  -  subtask  of  airspace 
observation.  This  subtask  has  always  be  performed 
when  flying  according  to  VFR  (Visual  Flight 
Rules)  in  order  to  avoid  collisions  with  other 
aircraft. 

As  missing  visual  attention  is  a  strong  indicator  for 
missing  situation  awareness,  and  missing  situation 
awareness  is  a  strong  contributing  factor  for 
accidents,  these  distributions  for  visual  attention 
measured  here  would  be  enough  reason  to  take 
corrective  actions.  Nevertheless  we  did  an  explicit 
test  in  the  simulator  by  introducing  collision 
aircraft.  They  flew  along  the  same  minimum  risk 
route  just  into  the  opposite  direction,  with  a  speed 
of  200  knots,  clearly  visible  in  the  outside  vision. 

According  to  signal  detection  theory,  e.g.  [Wickens 
1992],  after  the  detection  of  the  first  collision 
aircraft  there  would  be  a  strong  risk  of  a  complete 
change  of  the  attention  distribution.  Therefore  this 
event  “detection  of  collision  aircraft”  should 
reasonably  happen  only  once  per  pilot  without 
giving  any  hint  before. 

Because  of  the  statistical  difficult  low  number  of 
test  subjects,  the  pilots  were  asymmetrically 
divided  into  two  subgroups,  subgroup  I  with  ADI  / 
no  navigational  support  (4  pilots),  subgroup  II  with 
3D  /  no  navigational  support  (2  pilots).  At  the  end 
of  the  corresponding  flights  E  l  5  and  E  l  2  three 
successive  collision  aircraft  were  simulated.  After 
the  first  detection,  ascertained  by  callout,  avoidance 
maneuvers  or  clear  hints  in  the  eye  movement 
monitor,  the  experiment  was  terminated.  The  events 
“aircraft  detected”  and  “aircraft  not  detected”  had 
the  following  distribution: 


Subgroup 

Aircraft 

detected 

Aircraft  not 

detected 

I  ADI 

4 

1 

II  3D 

1 

5 

The  basic  hypothesis  H0  states  that  the  two  different 
technical  configurations  do  not  produce  a  different 
risk  of  colliding  with  another  aircraft.  A  Pearson- yl 
test  shows  a  significant  difference  with  paipha  = 
0.036,  but  because  the  actuarial  expectation  value 
per  cell  of  the  4 field  table  is  smaller  than  5,  it  is  not 
appropriate  in  this  case.  Luckily  the  side  sums  are 


almost  equal  and  due  to  the  experimental  design  a 
binomial  distribution  can  be  assumed,  therefor  the 
“single  sided  Fisher  Yates  exact  test”  can  be  used. 
This  value,  paipha  =  0.067,  is  not  significant  at  the 
confidence  level  used  for  scientific  experiments 
(95%),  but  due  to  the  lower  demands  of  the 
usability  paradigm,  e.g.  an  appropriate  confidence 
level  of  90%  suggested  by  [Nielsen  1993],  H0  can 
be  rejected  with  “strong  tendency  to  significance”. 

The  direct  transfer  of  this  result  from  the  small 
number  to  a  complete  population  of  pilots  is,  due 
to  the  design  of  the  experiment,  still  not  statistically 
valid  without  further  control.  Theoretically  this 
outcome  might  have  been  produced  by  a 
completely  different  visual  behavior  of  the  2 
“collision”  pilots  compared  to  the  4  “normal”  pilots 
and  the  total  population.  But  as  in  E_12,  the 
average  percentage  of  visual  attention  in  the  outside 
vision  is  14%  for  all  6  pilots,  compared  to  a  slightly 
smaller  12.5  %  for  the  two  “collision”  pilots,  there 
are  strong  hints  that  the  danger  of  not  detecting 
collision  aircraft  is  not  caused  by  interpersonal 
differences  but  by  the  configuration  of  displays. 

Discussion  of  the  technical  support 

Due  to  the  small  number  of  subjects  the  above 
mentioned  observations  and  results  just  have 
tendency  to  significance  (paiPha  <  0.1)  and  therefor  - 
according  to  classical  experimental  psychology  - 
want  to  be  used  with  caution.  Considering  the  lower 
statistical  demands  of  the  usability  paradigm,  e.g.  in 
[Nielsen  1993],  and  the  early  phase  of  the 
exploratory  process,  we  can  nevertheless  discuss 
the  following  findings: 

Each  of  the  described  levels  of  support  for  the 
navigation  subtask  improves  speed  and/or  quality  of 
performance. 

Intelligent  highlighting  using  the  situational 
knowledge  of  the  assistant  system  improves 
information  perception.  Additional  acoustic 
information  can  solve  captivation  of  the  attentional 
field  and  therefor  avoid  blind  areas,  as  E_7 
(Comparison  II)  shows.  Negative  effects  of 
cluttering  other  acoustic  information  sources,  which 
were  not  investigated  here  but  can  be  suspected, 
can  probably  be  avoided  by  nonvocal,  spatial 
coding  of  the  acoustic  signal. 

The  machine  generated  proposal  for  conflict 
resolution,  which  was  investigated  here,  is 
relatively  simple  due  needs  to  keep  the  experiment 
under  control.  In  situation  with  low  workload  pilots 
solve  these  conflicts  much  faster.  But  even  with 
that  simplicity,  in  situations  with  higher  workload, 
especially  with  an  additional  higher  frequency 
subtask  which  competes  for  concurrent  resources,  a 
computer-generated  proposal  clearly  improves 
speed  and  quality  of  conflict  resolution.  It  is  of 
course  mandatory,  beside  high  quality  and 
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reliability,  that  the  computer  solution  is  plausible 
and  transparent  in  order  to  build  up  appropriate 
trust  /  mistrust  and  therefor  enable  successful 
supervisory  control. 

The  simplified  activation  of  proposals  offers  an 
additional  speed  and  quality  improvement,  which 
can  be  used  optionally:  In  situation  with  sufficient 
resources,  pilots  can  choose  a  different,  more 
explicit  man  machine  communication  in  order  to 
maintain  situation  and  process  awareness,  in 
situations  with  lack  of  free  resources  pilots  can 
activate  very  simply  and  reliably  a  solution  that  is, 
at  least,  safe.  We  call  that  optional  aspect  “implicit 
support  of  operators’  own  resource  adaptation”  or 
“implicit  adaptation”.  The  machine  does  not 
explicitly  adapt  to  a  low  resource  situation,  but 
offers  implicit  means  for  resource  adaptation  (see 
also  [McKinley  1985],  [Verwey  1990]).  Few 
negative  effects  like  potential  risk  homeostasis  and 
complacency  have  been  observed.  They  have  to  be 
compensated  by  e.g.  supervised  training  (e.g.  with 
mission-replay  in  the  simulator). 

The  3D  display  with  an  information  fusion  for 
terrain,  flightpath  and  aircraft’s  attitude  offers 
benefits,  but  there  can  be  a  problem  with  the 
concentration  of  visual  resources  toward  the  head- 
down  displays.  This  effect,  in  these  experiments, 
led  to  a  clear  lack  of  situation  awareness  regarding 
collision  aircraft.  The  above  mentioned  simulator 
test  investigates  -  of  course  -  the  configuration 
without  navigation  support,  which  promised  to  be 
most  sensible  for  this  effect.  An  influence  of  the 
head  mounted  equipment  can  not  be  excluded,  the 
pilots  might  have  been  conditioned  to  a  simulator 
environment  where  there  was  no  experience  with 
collision  aircraft.  Moreover  this  concentration 
effect  will  be  of  quite  different  impact  with  a  two  or 
three  man  crew. 

Nevertheless  it  must  be  assured  that  the  existing 
risk  will  be  compensated.  Only  if  this  proves 
successful,  the  observed  clear  improvement  of 
flight  performance  can  fully  exploited.  The  freed 
resources  can  be  used  to  improve  other  subtasks 
like  navigation,  an  effect  which  will  be  even 
stronger  in  degraded  visual  conditions,  which 
where  not  investigated,  so  far. 

Discussion:  Is  eye  tracking  worthwhile? 

Eye  movement  measurement  offers  deep  insights 
into  man  machine  interaction  and  the  mental 
processes  of  pilots.  The  analysis  of  the  visual 
distribution  in  the  cockpit,  averaged  over  pilots  and 
time,  illuminates  global  effects  of  the  visual 
resource  with  high  qualitative  depth  and  face 
validity. 


own  resource  management  positively  or  negatively 
even  to  the  extreme  of  total  cognitive  fixation  to 
one  technical  subsystem.  A  direct  relationship 
between  the  risk  of  low  performance,  which  can 
often  not  directly  be  measured,  and  an  unfavorable 
visual  distribution,  which  can  be  measured,  clearly 
exists  and  can  be  used  to  detect  resource  based 
usability  problems  and  avoid  fatal  results. 

But  these  experiments  also  show  that  the  methods 
used  are  not  equally  sensitive  and  reliable  for  all 
ergonomical  questions.  There  are  quite  some 
examples  in  the  described  experiments  where  only 
one  method  succeeded  in  detecting  a  specific  fact 
while  the  others  were  insensitive.  A  holistic 
qualitative  picture  of  a  specific  man  machine 
interaction  seems  to  get  illuminated  best  with  an 
appropriate  combination  of  methods. 

Therefor  the  analysis  of  the  visual  resource  is  just 
one  additional,  but  powerful  tool  in  the  tool  box  of 
ergonomy.  Factors  like  time,  personal  effort  and 
money  will  contribute  to  the  decision  whether  this 
tool  will  be  used.  The  ongoing  development  of 
smaller  and  cheaper  hardware,  the  availability  of 
sophisticated  analysis  software  and  a  caSBAro  like 
high  integration  of  eyetracking  into  the  usability 
laboratory  will  make  it  easier  to  use  this  method  in 
the  development  process. 

Conclusion 

The  benefits  of  information  technology  ought  to  be 
exploited  also  for  battle  management  operations, 
but  we  know  that  there  might  be  side  effects  and 
new  risks  like  violations  of  the  human  limitations  of 
cognition  and  information  processing. 

There  are  methods  to  control  these  risks,  we  have  to 
use  these  methods  right  from  the  beginning  of  a 
development  process,  and  we  have  to  improve  these 
methods  permanently  in  order  to  catch  up  with  the 
speed  of  technology. 

Even  if  these  methods  are  no  guarantee  for  ideal 
information  systems,  they  offer  a  much  better 
chance  for  improving  usability.  If  we  do  not  take 
this  chance,  we  will  spend  money  on  new 
technology,  but  will  loose  systems  and  men  instead. 
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Visual  attention  is  a  limited  resource  and  has  to  be 
scheduled  by  the  pilots  to  different  information 
sources.  Technical  means  influence  this  operator’s 
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Foreword 

“[The]  ability  to  reach-back  and  use  capabilities  in  the 
continental  United  States  to  perform  functions  formerly 
accomplished  only  in  the  theater  of  military  operations 
is  one  of  the  highlights  of  operation  Allied  Force.  Such 
capability  improves  responsiveness  to  urgent 
requirements  in  a  conflict  and  reduces  the  amount  of 
equipment  and  the  number  of  personnel  that  must  be 
transported  to  the  theater.  In  short,  the  capability  to 
integrate  our  force  globally  yields  significant 
improvements  in  our  ability  to  respond  to  crises, 
particularly  during  their  initial  stages... 

Extensive  growth  in  communications  capacity  enabled 
an  unprecedented  degree  of  reliance  on  U.S. -based 
forces  to  provide  direct  support  for  in-theater  tasks. 
Targets  in  Kosovo  and  the  Federal  Republic  of 
Yugoslavia  were  developed  through  the  concerted 
effort  of  numerous  agencies  in  the  United  States 
cooperating  closely  with  commands  in  Europe. 
Planning  and  integration  of  cruise  missile  attacks  by 
bombers  operating  from  the  continental  United  States 
and  the  United  Kingdom  and  by  ships  and  submarines 
operating  in  the  Mediterranean  were  closely 
coordinated  by  commanders  and  planners  who  were 


widely  separated  geographically.  Bomb  damage 
assessments  of  strikes  made  against  targets  in  theater 
were  conducted  by  agencies  and  commands  located  in 
the  United  States  in  close  support  with  efforts  by 
commands  in  the  European  theater.  This  system  of 
using  geographically  dispersed  activities  to  perform 
and  integrate  bomb  damage  assessment  (BDA)  became 
known  as  federated  BDA.  Expert  personnel  located  in 
the  United  States  and  Europe  performed  detailed 
planning  of  information  operations.  Kosovo  operations 
continued  a  trend  of  increasing  global  integration  of 
U.S.  forces  and  commands  to  support  operations  in  a 
distant  theater. 

The  European  Theater’s  unprecedented  reliance  on 
organizations  and  personnel  in  the  United  States  and 
elsewhere  was  enabled  by  advances  in  information 
technology.  High-capacity  communications  made 
possible  the  exchange  of  large  amounts  of  data  such  as 
high-resolution  imagery  and  secure  video 
teleconferencing.  In  addition,  extensive  growth  and 
availability  in  defense  data  and  communications 
networks  enabled  unprecedented  coordination  by  staff 
members  in  European  commands  and  supporting 
commands  outside  of  Europe  by  secure  e-mail.  Secure 


Paper  presented  at  the  RTO  HFM  Symposium  on  “Usability  of  Information  in  Battle  Management 
Operations' held  in  Oslo,  Norway ,  10-13  April  2000,  and  published  in  RTO  MP-57. 
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high-capacity  networks  using  Web-based  technology 
permitted  personnel;  engaged  in  theater  to  access  up- 
to-date  information  posted  for  their  use  on  military 
Web  sites  around  the  world.”1 

This  paper  describes  how  collaboration  can  be  applied 
to  mission  processes  to  support  deliberate  and  crisis 
planning  and  operations.  Operation  Allied  Force 
operators  stated  that  proper  application  of  collaboration 
improved  the  effectiveness  of  information  processes, 
improved  product  quality  and  benefited  federated 
efforts  by  geographically  separated  partners.  During 
Operation  Allied  Force,  USEUCOM  operators 
demonstrated  that  collaboration  can  benefit  mission 
effectiveness.  Applying  collaboration  to  existing  or 
modified  mission  processes  needs  to  be  continued, 
refined  and  expanded  to  include  NATO  allies. 

This  paper  serves  as  a  reminder  of  the  most  important 
system  component,  the  military  operators,  who 
effectively  applied  collaboration  to  benefit  mission 
processes.  The  assistance  of  Lt  Col  Western,  LTC 
Steams,  LCDR  Kraft,  and  SMSgt  Schwarting  is 
appreciated.  The  dedicated  efforts  of  LCDR  Dodd  as 
operational  advisor  and  key  contributor  are  also 
recognized. 

Direct  comments  and  questions  about  this  paper  to  Mr. 
Greg  Chapin. 
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Introduction 

This  paper  describes  the  United  States  European 

Command’s  (USEUCOM’s)  application  of 

“collaboration2”  to  mission  processes.  The  main  points 

presented  are: 

1.  USEUCOM  is  judiciously  applying  collaboration 
to  benefit  mission  processes.  USEUCOM  is: 

•  Applying  collaboration  daily  to  mission  processes; 
not  just  talking  about  it 

•  Supporting  operations  and  contingency  planning 
with  collaboration  to  reduce  process  timelines  and 
improve  product  quality;  not  conducting 
demonstrations,  experiments,  or  studies 

•  Collaborating  in  an  operational  environment  with 
the  associated  constraints  and  security 
accreditation  requirements;  not  in  laboratories  or 
across  networks  using  equipment  not 
representative  of  USEUCOM’s  environment 

•  Benefiting  from  lessons  learned  and  best  practices 

•  Expanding  collaboration  when  and  where  it  makes 
sense 

According  to  Operation  Allied  Force  participants, 
collaboration  mitigates  the  effects  of  information 
overload,  improves  team  decision-making,  and 


2  “Collaboration”  is  more  than  just  the  technological  capabilities  (e.g. 
web-based  applications,  whiteboard,  text  chat,  and  audio).  For  this 
paper,  collaboration  includes: 

•  Technological  capabilities 

•  Collaborative  session  techniques 

•  Concept  of  operations  (e.g.,  process  owners,  roles  and 
responsibilities,  and  procedures) 

•  Standardized  product  templates 

Attachment  1  describes  the  primary  collaborative  techniques  and 
capabilities  used  at  USEUCOM. 
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synchronizes  situational  awareness.  USEUCOM 
participants  indicate  that  collaboration  is  positively 
impacting  mission  processes. 

2.  Operation  Allied  Force  demonstrated  the  benefits 
of  collaboration  and  highlighted  USEUCOM’ s 
operational  requirement  to  collaborate  with  NATO 
allies  to  support  operations.  As  a  result, 
USEUCOM  senior  leadership  is  advocating  the 
expansion  of  collaboration  within  the  intelligence 
community,  including  expansion  to  NATO  allies. 
NATO  should  consider  satisfying  the  collaboration 
requirement.  First,  NATO  might  consider 
satisfying  this  requirement  within  the  targeting 
community,  focusing  on  target  development  and 
nomination,  target  approval,  and  Air  Tasking 
Order  (ATO)  generation  and  management. 
Challenges  and  issues  addressed  by  USEUCOM  to 
satisfy  the  collaboration  requirement  are  discussed. 
Like  USEUCOM,  NATO  may  encounter  some 
similar  and  some  unique  challenges  and  issues. 

3.  USEUCOM  encountered  and  addressed  several 
challenges  and  issues  to  apply  collaboration 
successfully  to  mission  processes.  USEUCOM’s 
lessons  learned  and  best  practices  are  provided  for 
NATO’s  consideration  and  potential  use. 

USEUCOM  Experiences  Applying  Collaboration  to 
Mission  Processes 

“The  command,  control,  communications,  and 
computer  (C4)  support  to  Operation  Allied  Force  was 
highly  successful.  Several  important  communications 
capabilities  saw  their  first  significant  combat 
application:  use  of  Web-based  technologies  for 
coordination  and  information  sharing;  video 
teleconferencing  for  command,  control,  and 
coordination;  and  e-mail  for  coordination  and 
tasking.”3  This  section  describes  USEUCOM’s  use  of 
collaboration  for  combat  applications  during  Operation 
Allied  Force. 

USEUCOM  is  applying  collaboration  to  three  mission 
processes. 

•  Tomahawk  Land  Attack  Missile  (TLAM)  Mission 
Planning 

•  Final  Phase  of  Fixed  Targets  Development  and 
Nomination  for  Approval 

•  Synchronization  and  Sharing  of  Current 
Intelligence 

A  summary  of  each  process  describes: 

•  Need  and  Objective 

•  Process  and  Participants 


3  Kosovo/Operation  Allied  Force  After- Action  Report  to  Congress, 
31  January,  2000,  p.26 


•  Approach 

•  Results 

Comments  from  Operation  Allied  Force  participants  on 
applying  collaboration  to  USEUCOM’s  mission 
processes  conclude  this  section. 

TLAM  Mission  Planning 

Need  and  Objective: 

Before  applying  collaboration,  COMSIXTHFLT  tasked 
the  Cruise  Missile  Support  Activity  (CMSA)  with 
Tomahawk  Land  Attack  Missile  (TLAM)  mission 
planning  using  message  traffic  in  sequential  fashion. 
According  to  COMSIXTHFLT  and  CMSA 
participants,  the  ambiguity  and  uncertainty  inherent  in 
cumbersome,  text  only  message  tasking  needed  to  be 
replaced  by  a  more  efficient  tasking  mechanism  and 
process.  CMSA  frequently  contacted  COMSIXTHFLT 
for  additional  information  or  clarifications  to  satisfy  the 
Mission  Planning  Requests  (MPRs).  Participants 
indicated  that  this  methodology  needed  modification  to 
improve  accuracy,  effectiveness,  and  responsiveness  of 
TLAM  strikes. 

Process  and  Participants:  TLAM  mission  planning  is 
the  first  mission  process  in  USEUCOM  to  use 
collaboration  actively.  The  USEUCOM  process 
owner4  is  COMSIXTHFLT  TLAM  Strike  Cell,  Plans. 
Primary  participants  include: 

•  COMSIXTHFLT  Gaeta,  Italy:  TLAM  Strike  Cell, 
Plans 

•  U.S.  Joint  Forces  Command  (JFCOM)  Cruise 
Missile  Support  Activity  (CMSA)  Norfolk, 
Virginia 

Other  participants  include: 

•  Headquarters  United  States  European  Command 
(USEUCOM)  Stuttgart  Germany:  Targets,  Crisis 
Action  Plans 

Approach:  COMSIXTHFLT  provides  targeting  and 
mission  information  to  CMSA  by  completing  a  web- 
based  MPR  form.  CMSA  retrieves  the  MPR  from  the 
web  and  uses  the  information  to  work  the  task.  The 
web-based  MPR  form  with  pull  down  menus  improves 
the  communication  and  coordination  process.  The 
selectable  menus  facilitate  tasking  and  planning  by 
providing  a  mandatory  specific  standard  list  of  field 
declaration  options.  If  necessary,  CMSA  reviews 
certain  MPR  fields  with  COMSIXTHFLT  to  obtain 
clarifications  and  ensure  the  tasking  is  understood.  As 
the  tasking  organization,  COMSIXTHFLT  obtains 
relevant  imagery  products.  COMSIXTHFLT  and 


4  The  process  owner  is  the  responsible  for  creating  a  group  with  a 
mission  focus,  directing  the  participants,  and  controlling 
collaborative  sessions.  Collaborative  session  focus,  participant  roles 
and  responsibilities,  and  results  are  the  process  owner’s 
responsibility.  Reference  the  operational  category  of  the  Lessons 
Learned  and  Best  Practices  from  USEUCOM’s  Collaborative 
Experiences  section  for  recommended  process  owner  responsibilities. 
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CMSA  use  collaboration  to  simultaneously  review  and 
annotate  the  imagery  products.  The  simultaneous 
review  expedites  task  completion  and  synchronizes 
understanding.  CMSA  uses  the  MPR  and  collaborative 
session  results  to  fulfill  the  request  by  producing  the 
TLAM  mission  and  associated  TLAM  Target  Aimpoint 
Graphic  (TAG). 

Results:  COMSIXTHFLT  and  CMSA  participants 
indicated  that  applying  collaboration  and  using  the 
web-based  MPR  form  improved  the  TLAM  mission 
planning  process.  Participants  indicated  that  the 
modified  approach  resulted  in  improved  accuracy, 
effectiveness,  and  responsiveness  of  TLAM  strikes 
compared  with  the  former  approach.  The  MPR 
template  assisted  in  standardizing  terminology. 
Accuracy  and  completeness  in  satisfying  information 
requirements  improved.  TLAM  mission  planning 
participants  indicated  that  collaboration  removed 
tasking  ambiguity.  As  a  result,  participants  estimated 
that  response  times  from  mission  tasking  to  planning 
completion  were  nearly  cut  in  half.  COMSIXTHFLT’ s 
and  CMSA’s  continued  operational  use  and  advocacy 
for  the  web-based  MPR  form  and  collaboration  appears 
to  be  another  indication  of  success. 

Final  Phase  of  Fixed  Targets  Development  and 
Nomination  for  Approval 

“During  the  course  of  the  campaign,  NATO  developed 
mechanisms  for  delegating  target  approval  authority  to 
military  commanders.  For  selected  categories  of 
targets  —  for  example,  targets  in  downtown  Belgrade, 
in  Montenegro,  or  targets  likely  to  involve  high 
collateral  damage  —  NATO  reserved  approval  for 
higher  political  authorities.  NATO  leaders  used  this 
mechanism  to  ensure  that  member  nations  were  fully 
cognizant  of  particularly  sensitive  military  operations, 
and,  thereby,  to  help  sustain  the  unity  of  the  alliance. 

Legal  reviews  of  selected  targets  were  conducted  at 
successive  echelons  of  the  chain  of  command.  Targets 
nominated  for  approval  by  SACEUR  received  legal 
reviews  in  the  field.  Targets  nominated  that  met  the 
criteria  requiring  NCA  approval  received  detailed  legal 
scrutiny  by  the  Legal  Counsel  to  the  Chairman  of  the 
Joint  Chiefs  of  Staff  and  by  the  DOD  General  Counsel. 
Legal  reviews  involved  evaluation  of  certain 
targets  as  valid  military  targets  as  governed  by 
applicable  principles  of  the  laws  and  customs  of  armed 
conflict.”5  As  described  below,  the  USEUCOM  targets 
community  used  collaboration  to  support  portions  of 
the  fixed  targets  coordination  and  approval  process. 

Need  and  Objective:  The  extended  air  campaign 
against  Serbia  lasted  78  days  and  required  a  more 
efficient  targets  development  and  production  process. 


5  Kosovo/Operation  Allied  Force  After- Action  Report  to  Congress, 
31  January,  2000,  p.24 


“During  Operation  Allied  Force,  NATO  forces 
conducted  over  23,300  strike  missions  against  an  array 
of  targets.  These  strikes  were  directed  at  roughly  7,600 
target  aim  points  associated  with  a  variety  of  fixed 
targets  as  well  as  at  just  over  3,400  flex  targets  ”6  The 
USEUCOM  Chief  of  Targets  requested  that  the  fixed 
targets  development  and  nomination  process  be 
modified,  leveraging  collaboration  to  improve 
coordination  and  approval.  The  objective  was  to 
improve  process  efficiencies  to  increase  target 
availability  in  support  of  mission  objectives  and  strike 
operations. 

According  to  Operation  Allied  Force  collaborative 
session  participants,  the  process  used  prior  to  applying 
collaboration  is  important  to  understand  in  order  to 
fully  appreciate  the  benefits  gained  by  modifying  the 
process  to  use  collaboration.  Nine  geographically 
separated  sites  worked  on  and  coordinated  products 
sequentially.  One  site  forwarded  its  initial  work  as 
email  attachments,  message  traffic,  fax,  and/or  phone 
calls  to  other  sites  with  different  responsibilities. 
Another  site  made  product  changes  and  sent  the 
updates  to  participating  sites.  The  process  continued 
until  the  final  product  was  sent  to  decision-makers  for 
review  and  approval.  Decision-makers  received  an 
email  with  the  attached  product  information  and  either 
accepted  the  product  information  or  returned  it  for 
further  development. 

The  serial  workflow  extended  the  process  timeline  and 
provided  opportunities  for  the  nine  sites  to  introduce 
ambiguities  and  errors.  Participants  indicated  that 
communicating  point-to-point,  without  consensus  of 
other  participants,  created  confusion,  reduced  accuracy 
of  product  information,  and  caused  duplication  of 
efforts.  The  sites  involved  did  not  always  have  a 
thorough  understanding  of  other  sites’  tasks  and  goals. 
Therefore,  some  sites  only  understood  the  purpose, 
interdependencies,  and  value  of  their  contributions 
from  a  parochial  perspective.  Process  deficiencies 
made  execution  and  approval  too  time  consuming  and 
difficult  due  to  the  following: 

•  Redundant  information  flowing  to  decision-makers 

•  Sequential  coordination  and  approval  by  multiple 
organizations 

•  Maintaining  currency  of  information  and  products 
existing  in  multiple  versions  and  media  types 

•  Understanding  the  rationale  behind  changes  to 
avoid  repetitious  errors 

•  Tracking  the  status  of  products  held  for  refinement 
or  outstanding  action 

•  Inconsistent  quality  control  and  standardization 

Process  and  Participants:  The  collaborative  sessions 
supported  portions  of  the  overall  targeting  process. 


6  Kosovo/Operation  Allied  Force  After-Action  Report  to  Congress, 
31  January,  2000,  p.87 
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The  USEUCOM  targets  community  used  sessions  to 
construct  and  obtain  theater  agreement  on  final  target 
approval  briefings  for  decision-makers.  The 
USEUCOM  process  owner  is  the  Chief  of  Targets, 
Headquarters  European  Command  Targets  branch. 
Participants  included: 

•  HeadQuarters  United  States  European  Command 
(HQ  USEUCOM)  Stuttgart,  Germany:  Targets, 
Crisis  Action  Plans,  Judge  Advocate/Operations 
Law 

•  Joint  Task  Force  (JTF)  NOBLE  ANVIL  Naples, 
Italy:  Joint  Target  cell,  Judge  Advocate  (JA) 

•  United  States  Air  Force  Europe  (USAFE) 
Ramstein  AB,  Germany:  32nd  Air  Intelligence 
Squadron  (AIS)  Targets 

•  Joint  Analysis  Center  (JAC)  Royal  Air  Force 
(RAF)  Molesworth,  United  Kingdom:  JAC  Targets 

•  COMSIXTHFLT  Gaeta,  Italy:  TLAM  Strike  Cell, 
Plans  and  Targets 

•  Defense  Intelligence  Agency  (DIA)/Joint  Staff 
(JS)  J2T  Washington  DC:  Targets 

•  Combined  Air  Operations  Center  (CAOC) 
Vicenza,  Italy:  CAOC  Targets  US  representatives 

Approach:  Daily,  USEUCOM  and  US  national¬ 
targeting  agencies  held  regularly  scheduled, 
collaborative  sessions.  A  standardized  template 
containing  specific  target  information  required  for 
approval  aided  the  communication  and  coordination 
process.  The  template  standardized  terminology  and 
ensured  accurate  documentation  of  required  targeting 
information  prior  to  delivery  to  executive  decision¬ 
makers.  The  collaborative  sessions  focused  on 
targeting  information.  Participants  used  a  concept  of 
operations  with  roles  and  responsibilities  and  standard 
operating  procedures  to  properly  prepare  for  and 
control  sessions. 

The  Joint  Task  Force  (JTF)  targets  cell  led  the  sessions. 
Headquarters  USEUCOM  targets  branch  controlled  the 
target  information  during  sessions  and  monitored 
quality  control.  Intelligence  product  development  or 
other  functional  personnel  (e.g.,  legal  and  operations) 
contributed  to  or  reviewed  the  product  information. 
Collaboration  enabled  participants  to  view  imagery 
products  and  collate  existing  intelligence  information 
into  a  single  product.  Collaboration  allowed  the  target 
information  to  be  reviewed,  discussed,  modified,  and 
documented  concurrence  of  JTF  decisions  and  action 
items.  The  JTF  targets  cell  approved  or  held  target 
information  for  release  to  crisis  action  plans  branch. 
The  crisis  action  plans  branch  forwarded  the  target 
information  to  executive  decision-makers  for  final 
approval  or  provided  additional  requirements  during 
the  session. 

Results:  The  collaborative  sessions  successfully 
facilitated  the  coordination  and  approval  of  fixed 
targets  during  the  Operation  Allied  Force  air  campaign. 


USEUCOM  and  US  national  target  intelligence 
analysts  used  collaboration  effectively  to  share 
intelligence  information.  Collaboration  greatly  aided 
the  production  of  target  materials  used  by  senior  level 
decision-makers  (e.g.,  United  States  National 
Command  Authority  (NCA)  and  North  Atlantic 
Council  (NAC).  USEUCOM  targets  community 
representatives  estimate  that  the  timeline  decreased 
from  2-4  days  to  2-3  hours.  Participants  indicated  that 
coordination  and  synchronization  benefited,  improving 
process  efficiencies  and  product  quality.  The 
collaborative  sessions  required  detailed  target 
development  work  prior  to  convening  a  session.  This 
work  is  dependent  on  a  sufficient  number  and  right 
functional  combination  of  trained  personnel  with 
access  to  current  and  accurate  information.  The 
USEUCOM  targets  community  believes  the  potential 
exists  to  improve  the  quality  and  timeliness  of 
intelligence  throughout  the  targeting  process  by 
modifying  processes  to  include  collaboration  where 
appropriate. 

Collaboration  removed  or  reduced  the  process 
deficiencies  describe  under  Need  and  Objective. 

Posting  synchronized  information  to  web-sites  replaced 
multiple  email  attachments  that  contained  product 
information  and  modifications.  Simultaneous 
participation  improved  the  feedback  cycle.  As  a  result, 
the  rework  time  of  target  information  for  approval 
briefings  significantly  decreased  while  the  overall 
understanding  and  ultimate  value  of  the  information 
significantly  improved. 

Another  indication  of  success  is  the  USEUCOM  targets 
community  continued  use  and  refinement  of 
collaborative  sessions  to  support  operations  and 
contingency  planning.  Collaborative  sessions  are  now 
applied  from  beginning-to-end  for  target  development, 
nomination,  and  production  processes.  USEUCOM’ s 
dynamic  Area  of  Responsibility  (AOR)  required 
expansion  of  participants  since  Operation  Allied  Force. 
As  a  result,  eight  new  sites  now  participate  in 
collaborative  sessions. 

Synchronization  and  Sharing  of  Current  Intelligence 

Need  and  Objective:  The  decentralized,  independent, 
and  point-to-point  sequential  phone  coordination  of 
USEUCOM  intelligence  watches  often  results  in 
circular  or  unsynchronized  reporting.  The  Director  of 
Intelligence  and  HQ  USEUCOM  Watch  Chief 
requested  that  the  USEUCOM  watches  use 
collaboration  to  begin  working  together  as  a  single 
watch. 

Process  and  Participants:  The  USEUCOM  Watch 
community’s  collaborative  sessions  are  aimed  at 
enhancing  intra-theater  intelligence  watch 
coordination,  synchronization,  and  situational 
awareness  by  facilitating  a  single,  comprehensive 
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intelligence  picture  among  USEUCOM’s 
geographically  dispersed  intelligence  watches.  The 
Watch  community  holds  daily  sessions,  where  each 
organization’s  important  issues  are  presented  and 
discussed.  Ad-hoc  meetings  can  be  called  to 
coordinate  and  share  reports  on  high-interest  or  fast 
breaking  events.  Stations  from  other  theaters  or  from 
the  national  community  may  participate.  The  goal  is 
for  collaboration  to  become  the  primary  means  of 
coordination  among  USEUCOM  intelligence  watches. 

USEUCOM  had  to  identify  and  appoint  a  process 
owner.  The  intelligence  watches  across  USEUCOM 
did  not  have  a  theater-level  process  owner.  The 
Intelligence  Production  Chief  appointed  the  Chief  of 
the  Headquarters  European  Command  Watch  as  the 
process  owner.  Participants  include 

•  HQ  EUCOM  Stuttgart,  Germany:  HQ  Watch, 
Crisis  Action  Team  (CAT)  Watch  (when  active) 

•  Joint  Analysis  Center  (JAC)  Royal  Air  Force 
(RAF)  Molesworth,  United  Kingdom:  I&W  Watch 

•  United  States  Air  Force  Europe  (USAFE) 
Intelligence  Operations  Center  (IOC) 

An  expansion  plan  to  add  intelligence  watches  across 
USEUCOM  is  being  executed. 

Approach:  The  Director  of  Intelligence’s  daily  top 
issues  and  priorities  are  the  focus  for  sessions.  A 
concept  of  operations  and  standard  operating 
procedures  are  used  to  focus  and  control  watch 
sessions.  Headquarters  USEUCOM  Watch  leads  the 
session  and  shares  the  top  intelligence  issues  with 
participating  watches.  Each  participant  watch  obtains 
leadership’s  top  intelligence  issues  and  priorities, 
provides  an  update  status  on  each  issue,  and 
recommends  adding  issues.  Session  results  are 
documented  and  posted  to  a  web-site.  Addressing  fast¬ 
breaking  events,  developing  spot  reports,  and  working 
issues  together  using  collaboration  to  share  and  analyze 
intelligence  data  (e.g.,  maps,  imagery,  and  reports)  are 
planned. 

Results:  The  watch  collaborative  sessions  are  held 
daily.  USEUCOM  is  executing  the  plan  to  expand  site 
participation  and  insert  additional  collaborative 
techniques  and  capabilities  into  sessions.  For  example, 
watch  sessions  may  use  whiteboard  capabilities  to 
share  imagery,  review  and  adjust  indicator  lists,  and 
review  maps  and  charts  that  have  situation  overlays. 
The  daily  sessions  are  institutionalizing  a  collaborative 
mindset  and  are  providing  a  foundation  for  significant 
returns.  The  daily  review  and  coordination  of  the  top 
intelligence  issues: 

•  Ensure  awareness  of  the  Director  of  Intelligence’s 
top  issues  and  priorities 

•  Provide  components  and  Joint  Task  Forces  (JTFs) 
the  opportunity  to  modify  or  update  daily  issues 

•  Provide  a  forum  to  submit  new  issues 


•  Improve  synchronization  of  current  intelligence 
and  operations 

•  Expedite  situational  awareness  concerning 
developing  events 

•  Allow  non-participants  to  obtain  session  results 
from  a  web-site 

Participant  Comments  on  Applying  Collaboration  to 
USEUCOM’s  Mission  Processes 

As  USEUCOM’s  experiences  demonstrate, 
collaboration  can  mitigate  the  effects  of  information 
overload,  improve  team  decision-making,  and 
synchronize  situational  awareness.  Colloboration 
provided  one  means  to  execute  a  theater-federated 
process  with  worldwide  participants  as  described  in  the 
Report  to  Congress.  “A  federated  intelligence  process 
was  instituted  to  facilitate  burden-sharing  among 
intelligence  processing  centers  worldwide.  This 
approach  reduced  deployment  costs  while  maximizing 
the  use  of  existing  finite  resources.  The  federation 
process  was  highly  successful  and  depended  on 
information  sharing  and  agreements  among 
participants.  It  would  not  have  been  possible,  however, 
without  applied  technology,  innovation,  and  pre¬ 
planning  of  exercises.”  7 

According  to  Operation  Allied  Force  participants, 
collaboration  appears  to  be  positively  impacting  the 
coordination,  synchronization,  accuracy,  quality,  and 
timelines  of  USEUCOM  mission  processes. 
USEUCOM  experiences  using  collaboration 
demonstrated  some  of  the  phenomena  anticipated  by 
the  Joint  Vision  2010.  “  Joint  Vision  2010  anticipates 
these  phenomena  —  from  use  of  technologies  such  as 
video  teleconferencing  —  by  observing  ‘...higher 
echelons  will  use  these  technologies  to  reduce  the 
friction  of  war  and  to  apply  precise  centralized  control 
when  and  where  appropriate.  Real  time  information 
will  likely  drive  parallel,  not  sequential  planning  and 
real  time,  not  prearranged,  decision-making.  The 
optimal  balance  between  centralized  and  decentralized 
command  and  control  will  have  to  be  carefully 
developed  as  systems  are  brought  into  the 
inventories’.”8 

Participants  made  the  following  comments  on  the 
benefits  of  collaboration  to  USEUCOM’s  mission 
processes.  The  comments  are  divided  into  four 
categories. 

•  Process  Improvements 

•  Productivity  Improvements 

•  Product  Improvements 

•  Resource  Alternatives 


7  Kosovo/Operation  Allied  Force  After-Action  Report  to  Congress, 
31  January,  2000,  p.53 

8  Kosovo/Operation  Allied  Force  After-Action  Report  to  Congress, 
31  January,  2000,  p.28 
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Process  Improvements 

•  Provided  an  easy  method  to  keep  participants 
informed  on  current  process  and  product  status 

•  Raised  concerns  and  ensured  participants  knew 
process  steps 

•  Brought  various  disciplines  together  and  enabled 
cross-functional  (e.g.,  intelligence,  operations, 
legal)  process  participation 

•  Added  quality  to  the  product  review  process 

•  Facilitated  the  coordination  process  prior  to 
product  delivery  to  executive  decision-makers 

•  Provided  a  regularly  scheduled  forum  where 
thoughts  and  ideas  could  be  traded  based  on 
knowledge,  experience,  and  facts  that  each 
participant  provided 

•  Brought  time-sensitive  products  to  the  attention  of 
the  key  players  in  the  approval  process.  Reduced 
frequency  of  losing  products  in  the  approval  queue 
where  they  could  spend  weeks  before  being 
worked. 

•  Eliminated  serial  workflow  and  reduced  the 
number  of  product  modifications.  The  difficult 
and  convoluted  process  went  smoother  and  faster 
after  the  collaborative  sessions  began.9 

•  Provided  a  forum  for  sites  to  make  a  final, 
coordinated  check  and  provide  late-breaking 
updates  on  product  information 

Productivity  Improvements 

•  Provided  an  exponential  increase  in  productivity 

•  Decreased  estimated  production  time  significantly 

•  Reduced  the  discussion  and  lead  site’s  approval 
cycle  of  products 

•  Revolutionized  the  process,  significantly  reducing 
staff  effort 

Product  Improvements 

•  Advanced  the  development  of  a  standard  template 
for  presenting  product  information,  creating  a 
consistent  product  for  decision-makers 

•  Provided  a  single  product  template  that 
represented  each  product’s  information.  This 
template  provided  participants  with  a  common 
reference  point  that  resulted  in  better 
understanding  and  improved  team  decision¬ 
making. 

•  Synopsized  complex  product  information  and 
gave  participants  a  common  frame  of  reference  to 
facilitate  discussion.  Product  templates  were 
suitable  for  presentation  to  executive  decision¬ 
makers,  further  streamlining  the  approval  process. 


9  Participants  realized  that  the  serial  decision  process  is  not  necessary 
given  up-front,  collaborative  coordination.  Once  a  primary  decision 
is  made,  many  subsequent  actions  do  not  require  additional  decisions 
to  be  made.  The  decisions  only  need  to  be  executed. 


Resource  Alternatives 

•  Provided  senior  to  mid-level  decision-makers  and 
operators  (e.g.,  targeting  and  analytical)  an 
alternative  to  video  teleconferences  (VTCs)  that 
were  dominated  by  flag-level  officers 

•  Let  operators  participate  from  their  workspaces 
and  allowed  access  to  key  information  and 
materials  during  sessions 

Collaboration  with  NATO  Allies 

“Although  experience  in  Operation  Allied  Force 
confirmed  that  the  United  States  and  our  allies  have 
made  significant  accomplishments  working  together,  it 
also  made  clear  that  improvements  are  necessary. . . 
Among  the  most  important  of  these  are  deficiencies  in 
command-and-control  and  information  systems,  secure 
communications,  precision  strike  capability,  air 
operations  support,  and  mobility  systems.  During 
Allied  Force  these  shortcomings  . . .  impeded  our  ability 
to  operate  more  effectively  with  NATO  allies.”10 

This  section  discusses  the  requirement, 
recommendation,  and  challenges  associated  with 
improving  collaboration  between  NATO  allies  to 
support  operations . 

1.  Operation  Allied  Force  demonstrated  that 
collaboration  with  NATO  allies  is  necessary. 
Several  USEUCOM  military  operators  validated 
that  collaboration  with  NATO  allies  is  an 
operational  requirement. 

2.  USEUCOM  recommendations  are  provided  for 
NATO’s  consideration  in  satisfying  the 
requirement  to  improve  collaboration  between 
NATO  allies 

3.  Challenges  and  issues  faced  by  USEUCOM  to 
apply  collaboration  successfully  are  described. 
NATO  may  encounter  some  similar  and  some 
unique  challenges  and  issues. 

Collaboration  with  NATO  Is  an  Operational 
Requirement 

Operation  Allied  Force  clearly  demonstrated, 
collaboration  with  NATO  allies  is  necessary.  One 
lesson  learned  by  the  USEUCOM  targets  community  is 
that  increased  and  improved  collaboration  between 
NATO  allies  to  support  the  targeting  process  is  needed. 
As  a  result,  USEUCOM  operators  have  stated  the 
requirement  to  collaborate  with  NATO  to  support 
operations.  Since  USEUCOM  is  a  participant  in  NATO 
operations,  a  reliable  collaborative  capability  with 
NATO  counterparts  is  essential. 

Several  USEUCOM  operators  stated  that  improved 
collaboration  with  NATO  allies  is  a  top  priority. 


10  Kosovo/Operation  Allied  Force  After-Action  Report  to  Congress, 
31  January,  2000,  p.xix 
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Intelligence  operations  and  geo-spatial  information 
sendees  personnel  validated  the  requirement  to 
collaborate  with  NATO  allies.  A  US  representative  at 
the  Combined  Air  Operations  Center  (CAOC)  targeting 
cell  stated  the  requirement  to  allow  collaboration 
across  security  enclaves  (SECRET  US  ONLY  to 
SECRET  RELEASABLE  TO  NATO).  NATO  should 
continue  or  begin  to  address  the  challenges  and  issues 
with  increased  emphasis  and  additional  resources  to 
improve  or  replace  existing  capabilities  between 
NATO  allies  and  satisfy  the  collaboration  requirement. 

Recommendations  to  Satisfy  the  Collaboration 
Requirement 

Based  on  Operation  Allied  Force  lessons  learned, 
NATO  should  consider  satisfying  the  collaboration 
requirement.  First,  NATO  might  consider  satisfying 
this  requirement  within  the  targets  community  focusing 
on  target  development  and  nomination,  target  approval, 
and  Air  Tasking  Order  (ATO)  generation  and 
management. 

Tasking  an  existing  or  new  NATO  working  group  to 
work  the  collaboration  requirement  is  offered  for 
NATO’s  consideration.  The  working  group  could: 

•  Coordinate  with  NATO  and  member  nation 
operators  to  identify  a  process  owner  and  develop 
a  Concept  of  Operations 

•  Develop  Standard  Operating  Procedures  and 
provide  training 

•  Work  with  NATO  network  domains  and  site 
technical  representatives  to  develop  an  architecture 
that  maximizes  interoperability  and  integrates  into 
the  enterprise  baseline  and  participant  sites’ 
system  baselines 

•  Work  with  information  security  representatives  to 
provide  procedural  and  technical  solutions  that 
meet  accreditation  and  security  risk  management 
requirements 

Challenges  and  Issues  to  Satisfy  the  Collaboration 
Requirement 

Like  USEUCOM,  NATO  will  encounter  challenges 
and  issues  to  satisfy  the  collaboration  requirement. 
The  working  group  will  need  to  address  the  NATO  and 
Member  Nations’  challenges  and  issues  encountered. 
USEUCOM  has  encountered  and  addressed  several 
challenges  and  issues  that  NATO  may  encounter. 
Some  challenges  and  issues  faced  by  NATO  may  be 
more  complex  than  or  different  from  USEUCOM’s 
experiences  since  NATO  is  a  supra-national 
organization  comprised  of  multiple  sovereign  nation¬ 
states.  Some  of  the  challenges  and  issues  that  may  need 
to  be  addressed  are: 

Policy:  Leadership,  representing  both  NATO  and 

member  nations,  must  identify  what  information  and 


products  can  be  shared  during  collaborative  sessions. 
Otherwise,  releaseability  issues  may  reduce  the  benefit 
collaboration  can  have  on  time  sensitive  mission 
processes.  Leadership  must  balance  the  risk 
associated  with  conducting  NATO  collaborative 
sessions  against  the  potential  consequences  to 
personnel  and  mission  if  information  is  not  shared  in  a 
timely  fashion.  Collaboration  and  information  sharing 
is  always  a  risk  and  requires  investments  of  time,  staff, 
and  other  resources.  Leadership  must  determine 
whether  the  investment  and  risk  of  sharing  information 
is  worth  the  potential  returns.  A  compromise  between 
the  “need  to  know”  and  “need  to  share”  policies  needs 
to  be  made. 

For  example,  U.S.  information  releaseability  policy 
may  inhibit  collaboration  with  NATO  unless 
modifications  are  made.  “In  addition  to  dissemination 
problems  on  the  data  networks  discussed  above, 

U.S.  sensitivity  to  releasing  certain  types  of 
information  greatly  inhibited  combined  planning  and 
operations  in  some  areas.”11 

The  Report  to  Congress  addresses  facilitating 
distribution  of  U.S.  intelligence  products  to  warfighters 
and  allies.  “Much  of  the  U.S.  information  in  question 
should  be  classified  at  the  SECRET  collateral  level 
releasable  to  the  coalition  operation  so  that  it  can  be 
effectively  used  by  both  U.S.  and  coalition  warfighters. 
To  the  extent  possible,  imagery  and  signals  intelligence 
data  should  classified  ‘SECRET/NOFORN  Releasable 
to  NATO,’  and  sources  and  methods  should  be 
protected  'by  exception,’  rather  than  the  other  way 
around.”12  “The  Department  will  explore  ways  to 
permit  intelligence  and  other  information  to  be 
classified  at  the  lowest  possible  classification  level  in 
order  to  ensure  its  availability  to  warfighters  and 
coalition  partners,  while  still  protecting  intelligence 
sources  and  methods. ”lj 

Process  Owners  and  Concept  of  Operations 
(CONOPs):  The  identification  and  appointment  of  a 
process  owner  with  existing  or  formally  announced, 
delegated  executive  authority  that  is  recognized  by  all 
participant  sites  may  be  more  complex  since  NATO  is 
a  supra-national  organization.  Likewise,  a  dedicated, 
cooperative  effort  will  be  required  to  develop  a  useful 
CONOPs  . 

Infrastructure  and  Interoperability:  The  network 
connectivity  and  information  infrastructure  required  for 
collaboration  needs  to  be  provided  to  participant  sites. 
USEUCOM’s  collaborative  efforts  leveraged  existing 
U.S.  infrastructure  and  connectivity.  General  systems 


11  Kosovo/Operation  Allied  Force  After-Action  Report  to  Congress, 
31  January,  2000,  p.50 

12  Kosovo/Operation  Allied  Force  After-Action  Report  to  Congress, 
31  January,  2000,  p. 51 

13  Kosovo/Operation  Allied  Force  After-Action  Report  to  Congress, 
31  January,  2000,  p.134 
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engineering  expertise  will  be  required  to  assist  with  the 
planning  and  management  of  the  supra-enterprise 
architecture. 

The  Report  to  Congress  highlights  problem  areas. 
“Although  successful  in  some  areas,  NATO 
[command,  control,  communications,  and  computers] 
C4  capability  was  limited  by  the  lack  of  C4  agreements 
and  the  need  for  more  stringent  enforcement  and 
implementation  of  existing  agreements.  Problem  areas 
included  (1)  sharing  of  bandwidth  and  C4  assets, 

(2)  C4  network  integration  training  standards  at  the 
combined  and  joint  task  force  level,  (3)  spectrum 
management  within  combined  and  joint  task  forces, 

(4)  network  security,  (5)  lack  of  timely  compliance 
with  NATO  standardization  agreements  (STANAGs), 
and  (6)  releasability  of  information.  In  addition,  the  C4 
host  nation  agreement  process  needs  to  be  expedited, 
and  the  focus  of  the  agreements  should  be  on  standards 
and  architectures  rather  than  specific  hardware.”14 

“Information  interoperability  was  sometimes  a  major 
problem.  This  was  true  during  both  U.S.  joint 
operations  and  combined  NATO  operations. 
Interoperability  concerns  were  noted  in  how 
information  is  disseminated  (the  supporting  C4 
infrastructure)  and  how  to  disseminate  it  securely 
(releasability  of  various  levels  of  classification). 
Dissemination  networking  and  procedures  were  ad  hoc, 
and  it  was  never  possible  to  present  a  common 
operational  picture  to  joint  and  allied  commanders. . .  In 
summary,  we  see  that  interoperability  will  be  the 
cornerstone  for  future  alliance  participation.”15 

NATO  and  U.S.  are  providing  mechanisms  to  assist  in 
formalizing  command,  control,  communications  and 
computers  (C4).  Formal  C4  policies  will  likely  benefit 
efforts  to  provide  a  collaborative  infrastructure  by 
addressing  needs  documented  in  the  Report  to 
Congress.  “As  the  United  States  and  NATO  fielded 
these  capabilities,  some  policy  differences  emerged 
that  highlighted  the  need  for  increased  emphasis  and 
coordination  in  the  alliance.  The  Defense  Capabilities 
Initiative  and  NATO’s  Strategic  Concept  provide 
mechanisms  to  assist  in  formalizing  C4  policies. 
Intensive  efforts  in  this  vital  area  of  alliance  command, 
control,  communications,  and  computers  will 
contribute  to  improved  interoperability  and  reduction 
in  the  imbalance  in  capabilities.”16 

One  possibility  is  to  use  NATO’s  existing  network 
domains  as  a  starting  point.  The  four  key  NATO 
intelligence  network  domains  should  be  considered  in 


u  Kosovo/Operation  Allied  Force  After-Action  Report  to  Congress, 
31  January,  2000,  p.47,  48 

15  Kosovo/Operation  Allied  Force  After-Action  Report  to  Congress, 
31  January,  2000,  p.49,  51 

16  Kosovo/Operation  Allied  Force  After-Action  Report  to  Congress, 
31  January,  2000,  p.26 


the  analysis  and  development  of  solutions.  The  four 
key  NATO  intelligence  network  domains  are: 

•  The  NATO-nations  Battlefield  Information  and 
Collection  Exploitation  Systems  (BICES) 

•  Allied  Command  Europe’s  (ACE)  ACE  Command 
and  Control  Information  System  (ACCIS)  centered 
around  CRONUS,  with  its  intelligence 
applications 

•  Allied  Command  Atlantic’s  (ACLANT)  Maritime 
Command  and  Control  Information  System 
(MCCIS)/National  Intelligence  Data  Transfer 
System  (NIDTS),  with  its  intelligence  applications 

•  United  States  European  Command’s  (USEUCOM) 
Linked  Operations-Intelligence  Centers  Europe 
(LOCE) 

Other  NATO  networks  that  are  used  by  different 
functional  communities  should  probably  also  be 
considered.  Operation  Allied  Force  demonstrated 
clearly  the  need  and  benefits  of  having  the  required 
combination  of  functional  representatives  (e.g., 
intelligence,  operations,  legal)  participate  in 
collaborative  sessions. 

The  Intelligence  Projects  Integrated  Working  Group 
(IPIWG)17  continues  to  discuss  issues  such  as  the  use 
of  collaboration  tools.  Web  access  and  email 
capabilities  are  already  available  between  NATO 
domains  and  partially  satisfy  the  collaboration 
requirement.  BICES  is  currently  evaluating  and 
attempting  to  use  some  collaborative  capabilities. 
NATO  could  benefit  from  the  International  Military 
Staff  (IMS)  Intelligence  Division  and  the  IPIWG 
working  together  on  collaboration.  This  group  could 
build  on  the  existing  network  domains  to  provide  an 
initial  set  of  collaborative  capabilities. 

Culture:  Differences  in  culture,  language,  automation 
skills,  as  well  as  experience  in  collaboration  and 
information  sharing  may  impact  mission  groups  in 
NATO  more  than  USEUCOM.  Symbology, 
terminology,  and  language  usage  need  to  be 
standardized. 

Funding:  The  acquisition  approach  and  resources  to 
provide  collaboration  and  the  associated  dependencies 
(e.g.,  infrastructure,  operational  and  technical  support, 
and  training)  must  be  determined. 


17  SHAPE’S  ACE  Intelligence  Architecture  Working  Group 
(AIAWG)  created  the  Intelligence  Projects  Integration  Working 
Group  (IPIWG)  and  tasked  it  to  develop  a  feasible,  Near-to-Mid 
Term  NATO  Wide  Intelligence  Architecture  taking  into  account 
existing  systems  and  ongoing  projects.  The  four  major  NATO 
intelligence-related  systems  (or  domains)  listed  above  are  part  of  this 
effort. 
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Lessons  Learned  and  Best  Practices  from 
USEUCOM’s  Collaborative  Experiences 

USEUCOM  experienced  several  overarching  lessons 
learned  and  improvements  needed  that  are  documented 
in  the  Report  to  Congress.  . .  [T]he  Department  needs 
to  further  develop  and  refine  tactics,  techniques,  and 
procedures  for  federated  intelligence  efforts  and  to 
reassess  and  size  long-haul  communications  needs 
accordingly.  Planning  for  intelligence  communications 
needs  must  include  deployable  systems  and 
technicians.  Additionally,  the  Department  needs  a 
clear  policy  and  implementation  plan  to  explain  when 
and  how  coalition  partners  can  be  connected  to  U.S. 
networks  and,  when  and  how  data  can  be  shared  with 
those  partners.”18 

“The  widespread  use  of  video  teleconferencing  and 
other  advanced  technologies  for  command  and  control 
and  collaborative  planning  presented  numerous 
limitations  and  challenges.  In  order  to  optimize  the 
application  of  these  systems  and  accustom  operational 
commanders  to  their  effects,  appropriate  doctrine, 
tactics,  techniques,  and  procedures  must  be  developed. 
In  addition,  these  technologies  should  be  included 
regularly  in  future  large-scale  joint  and  combined 
training  exercises.”19  USEUCOM’s  lessons  learned 
and  best  practices  using  collaboration  may  assist  in 
optimizing  the  application  of  collaboration  and 
accustomizing  operational  commanders  to  the  effects 
of  collaboration. 

Lessons  learned  and  best  practices  from  USEUCOM’s 
experiences  are  provided  for  NATO’s  consideration 
and  potential  use  to  satisfy  the  collaboration 
requirement.  Lessons  learned  and  best  practices  are 
grouped  into  three  categories. 

•  Technical 

•  Operational 

•  Accreditation  and  Approval  to  Operate 

Technical  Lesson  Learned  and  Best  Practices 

According  to  Operation  Allied  Force  participants,  the 
performance,  reliability,  and  simplicity  of  the 
collaborative  capabilities  within  the  operational 
environment  are  the  primary  factors  that  affect  operator 
acceptance  and  use.  The  technical  lessons  learned  and 
best  practices  that  assisted  in  providing  collaboration 
capabilities  with  acceptable  performance,  reliability, 
and  simplicity  in  the  USEUCOM  operational 
environment  are  provided  below. 

Basic  Capabilities:  Capabilities  need  to  be  simple  to 
allow  operators  with  basic  computer  skills  to 


18  Kosovo/Operation  Allied  Force  After-Action  Report  to  Congress, 
3 1  January,  2000,  p.53 

19  Kosovo/Operation  Allied  Force  After-Action  Report  to  Congress, 
31  January,  2000,  p.  xxii 


participate.  A  trade  off  between  complex  capabilities 
and  keeping  the  system  simple  should  be  made  with 
military  operators.  Use  of  capabilities  will  probably 
evolve  as  the  process  matures  and  increased  benefits 
are  realized. 

Network  Infrastructure  and  Architecture:  Collaboration 
is  dependent  on  network  infrastructure  and 
connectivity  to  mission-essential  participants.  The 
stability,  network  capacity  (bandwidth),  and 
configuration  of  the  network  infrastructure  directly 
impact  the  performance  and  reliability  of  collaborative 
sessions.  Network  capacity  should  be  viewed  as  an 
operational  resource.  Operators  should  perform  a  cost- 
benefit  analysis  that  compares  the  mission  benefits  of 
collaborative  capabilities  with  other  capabilities  and 
requirements  that  consume  bandwidth.  An  optimal 
mixture  of  network  infrastructure,  design  of  the 
collaborative  capabilities,  and  procedural  techniques 
are  required  to  maximize  performance  and  reliability. 

The  impact  to  network  bandwidth  needs  to  be  assessed. 
Accreditation  and  the  “approval  to  operate”  are  based 
on  this  assessment.  Network  impact  is  difficult  to 
assess  since  items  that  consume  bandwidth  change 
dynamically  during  a  collaborative  session.  It  is 
similar  to  asking  what  is  the  impact  to  network 
bandwidth  of  email  with  attachments  or  Internet 
activity. 

Some  variables  that  affect  performance,  reliability  and 
required  network  bandwidth  are: 

•  Enterprise  server  location  based  on  an  analyses  of 
several  items  listed  below 

•  Physical  location  of  participants 

•  Number  of  simultaneous  participants  from  Local 
Area  Network  LAN,  from  Wide  Area  Network 
(WAN) 

•  Concept  of  operations  (e.g.,  roles  and 
responsibilities) 

•  Frequency  of  server  access  and  file  transfers  across 
LAN  and  WAN 

•  Information  structure  and  movement  within  the 
process 

•  Technical  support 

•  Collaborative  techniques  and  procedures  used 
(e.g.,  single-point  application  sharing  vs.  giving 
control  to  multiple  different  sites,  sharing  multiple 
small  files  or  one  large  file) 

•  Collaborative  capabilities  used 

•  Size  of  information  being  shared 

•  Configuration  of  enterprise  servers  and 
workstation  clients  (e.g.,  audio  codec  selected) 

•  Network  architecture,  management  control,  and 
distribution  path  alternatives 

The  mission  process  using  collaboration  should  be 
compared  with  the  old  process  as  a  way  to  view  and 
assess  the  overall  impact  to  network  capacity.  For 
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example,  the  modified  process  using  collaboration 
eliminated  numerous  emails  with  attachments  sent  by 
multiple  sites  to  multiple  sites  during  the  old  process. 
This  network  capacity  savings  should  be  included  in 
the  overall  assessment. 

Integration:  Integration  of  collaborative  capabilities 
into  the  enterprise  baseline  and  into  each  participant 
site’s  local  system  baseline  is  recommended. 
Capabilities  should  require  few  modifications  to 
existing  baselines.  Requiring  a  separate  workstation  for 
collaboration  in  existing  spaces  is  not  acceptable.  The 
collaborative  capabilities  should  work  on  existing 
workstations  with  minimal  modifications. 
Collaborative  capabilities  should  be  independent  of 
workstation  make  and  operating  system.  Workstation 
make  and  operating  system  independence  decreases 
investment  costs  and  increases  the  probability  of  new 
participants  quickly  joining  the  collaborative 
architecture  at  garrison  and  deployed  sites. 

The  collaborative  capabilities  (e.g.,  text  chat,  voice 
audio),  mission  applications  (e.g.,  presentation  and 
imagery  software),  site  information  repository  (e.g., 
local  or  network  drives  where  information  is  stored), 
and  software  product  templates  were  integrated  into  the 
existing  enterprise  or  sites’  system  software  baselines. 
Collaborative  capabilities  must  be  developed  to  work 
within  the  designated  environment’s  system  baseline 
configuration.  The  collaborative  capabilities  must 
interoperate  with  the  network  infrastructure,  software 
(e.g.,  browser  profiles,  security  permissions,  and  user 
account  privileges)  and  hardware  (e.g.  sound  cards) 
baselines.  Collaborative  capabilities  with  a  high 
degree  of  integration  into  or  interfaces  with  the 
enterprise  or  sites’  system  baselines  should  be  resilient 
as  possible  to  baseline  component  configuration 
settings  and  system  baseline  changes. 

Collaborative  session  participants  continue  to  access 
their  local  site’s  information  repository,  from 
workstation  or  site  network  drives.  No  additional  effort 
is  required  to  retrieve,  store,  and  post  or  disseminate 
information.  Participants  use  the  site  information 
repository  since  the  familiar  structure  allows 
information  to  be  located  efficiently.  Participants 
develop  and  share  products  by  using  local  mission 
applications  software.  Integration  significantly  reduces 
application  and  information  repository  training 
requirements. 

Acceptance  Testing:  Testing  the  collaborative 
capabilities’  performance  and  reliability  in  the 
operational  environment  prior  to  acceptance  is 
imperative.  Collaborative  capabilities  may  meet  a 
specification  and  perform  well  in  a  laboratory 
environment.  However,  the  system  may  not  integrate 
easily  or  well  into  the  operational  enterprise  baseline 
and  participating  sites’  baselines. 


Interoperability:  Collaborative  capabilities  should  be 
interoperable  with  enterprise  and  participating  sites’ 
mission  application  software.  Proprietary  software  or 
equipment  should  be  avoided.  Adhering  to  commercial 
international  standards  increases  the  probability  of 
achieving  interoperability.  Using  mainstream 
commercial  products  improves  the  ability  to  evolve 
with  future  technologies  while  providing  operators 
capabilities  today. 

Configuration _ Management:  Configuration 

management  and  testing  need  to  be  approached  from  a 
mission  group  perspective.  The  mission  group 
perspective  consists  of  interdependent  baselines. 
Configuration  management  and  testing  of 
interdependent  baselines  present  unique  challenges 
compared  with  traditional  independent  baselines. 
Interfaces  and  dependencies  of  the  interdependent 
baseline  must  be  identified  and  tested  when 
collaboration  dependent  participant  sites’  or  enterprise 
baseline  configuration  items  are  modified.  For 
example,  one  participant  site  may  upgrade  the  browser 
component  of  its  baseline.  The  browser  upgrade  may 
make  some  collaborative  capabilities  (e.g.,  audio,  and 
text  chat)  incompatible  with  the  existing  enterprise 
capability  used  by  the  mission  process  group. 
Therefore,  the  site’s  baseline  must  be  made  compatible 
with  the  mission  process  group’s  capabilities. 

Technical  Support:  Technical  support  is  required  on  a  7 
day  by  24-hour  basis  during  operations.  Technical 
support  representatives  need  to  assist  operators  and 
team  with  local  technical  personnel.  Local  technical 
personnel  are  often  required  to  isolate  and  resolve 
technical  issues  manifested  as  symptoms  when  using 
enterprise  collaborative  capabilities.  The  symptoms 
could  be  caused  by  the  local  site’s  system  baseline  or 
network  infrastructure  and  only  experienced  when 
using  collaborative  capabilities. 

Professional  Relationships:  Integration  of  collaborative 
capabilities  into  the  enterprise  and  sites’  baselines  to 
achieve  good  performance  and  reliability  depends  on 
professional  relationships  developed  between  general 
systems  engineering  and  site  technical  and  security 
personnel;  and  between  general  systems  engineering 
representatives  and  military  operators.  Good 
professional  relationships  between  participating  sites’ 
military  operators,  technical  and  security  personnel  are 
instrumental  in  successfully  achieving  the 
implementation  of  the  collaborative  capabilities. 

Operational  Lesson  Learned  and  Best  Practices 

Technical  lessons  learned  and  best  practices  are  not 
sufficient  to  realize  the  potential  benefits  of 
collaboration.  Operational  lessons  learned  and  best 
practices  must  complement  the  technical  ones. 
Operation  Allied  Force  lessons  learned  demonstrate 
that  collaboration  can  benefit  mission  effectiveness 
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when  judiciously  applied  to  existing  or  modified 
processes,  a  process  owner  is  appointed  who  carries  out 
the  recommended  responsibilities  listed  below,  and  a 
Concept  of  Operations  (CONOPs)  is  documented. 
Collaboration  does  not  replace  the  need  for  the  right 
functional  combination  of  well-trained,  prepared 
personnel  who  have  access  to  current  and  accurate 
information.  Operational  lessons  learned  and  best 
practices  that  contributed  to  using  collaboration 
successfully  during  Operation  Allied  Force  are 
provided  below. 

Appointment  of  Process  Owner:  The  command  or 
enterprise  must  identify  and  appoint  a  process  owner 
with  existing  or  formally  announced,  delegated 
authority  and  responsibility  that  is  recognized  by  all 
participant  sites.  A  process  owner  must  be  appointed 
for  each  collaborative  mission  group,  mission  process, 
or  routine  collaborative  session,  as  appropriate.  The 
process  owner  may  appoint  an  operational  advisor  to 
assist  with  these  responsibilities.  Recommended 
process  owner  responsibilities  are: 

•  Must  understand  the  entire  process  and  the  current 
operation  in  detail 

•  Own  and  provide  mission  group’s  requirements 

•  Identify  participants  and  sites 

•  Coordinate  with  and  assist  participant  sites  to  get 
capabilities  funded  and  implemented 

•  Provide  military  direction,  guidance,  and 
information  to  participants 

•  Develop  Concept  of  Operations  (CONOPs)  (e.g., 
how  to  use  when  and  where  to  meet;  define  and 
assign  roles  and  responsibilities  before,  during, 
and  after  session.) 

>  Identify  what  sites  will  have  the  leader, 
information  coordinator,  and/or  production 
developer  role(s). 

•  Provide  network  connectivity  requirements 

•  Identify  process  and  products,  for  potential 
modification  with  collaboration 

•  Develops  the  Standard  Operation  Procedures 
(SOPs).  The  SOPs  document  session  step-by-step 
specific  instructions  for  each  participant.  The  SOP 
development  is  instrumental  in  modifying  the 
process,  selecting  and  developing  collaborative 
techniques,  capabilities,  and  standardized  product 
templates. 

Concept  of  Operations  (CONOPs):  Defining  roles  and 
responsibilities  and  preparing  for  the  collaborative 
sessions  is  important  to  achieve  success.  The  process 
owner  or  delegate  leads  this  effort  with  the  assistance 
of  the  general  systems  engineer.  Session  roles  (e.g., 
leader,  information  coordinator,  product  developer)  and 
responsibilities  assigned  take  into  account  the  process 
timeline,  experience,  control  desired,  command 
structure,  and  number  of  participating  sites. 
Participants  have  the  capability  and  opportunity  to 
provide  local  expertise.  The  interdependent  roles  and 


responsibilities  require  definition  for  the  “before”, 
“during”,  and  “after”  phases  of  the  collaborative 
session.  These  roles  and  responsibilities  are  critical  to 
establish  and  execute  successful  sessions.  Roles  and 
responsibilities  of  federated  partners  must  also  be 
defined  and  documented.  All  sites,  except  site(s)  with 
the  lead  role,  participate  for  coordination  purposes. 
Examples  of  three  key  roles20  are  listed  below. 

Leader :  The  leader  works  with  the  information 
coordinator  to  ensure  preparation  and  execution  of 
collaborative  sessions. 

Before  Session 

•  Determine  what  items  to  review  and  assign 
preparation  responsibilities 

•  Identify  products  to  review  and  revisit  during 
session 

•  Inform  sites  of  key  personnel  and  functional  skill 
mix  required  during  session 

•  Develop  agenda  and  set  schedule 
During  Session 

•  Focus  on  running  the  session  and  obtaining  results 
in  a  reasonable  time  frame 

•  Task  development  work  and  issue  priorities 

•  Assign  action  items  and  suspenses 

•  Chair  session  in  close  coordination  with 
information  coordinator 

•  Approve  or  hold  information  and  product  release 

•  Act  as  final  authority  on  questions  and  decisions 
After  Session 

•  Write  and  provide  summary  of  session  and  actions 
to  participants 

Information  Coordinator :  The  information  coordinator 
works  with  the  leader  to  ensure  preparation  and 
execution  of  collaborative  sessions.  The  information 
coordinator  is  responsible  for  information  and  product 
management. 

Before  Session 

•  Schedule  session  time  and  setup  conference  on 
server 

•  Coordinate  agenda  with  session  lead  site 

•  Test  participant  sites’ systems 

•  Gather  product  information  for  sessions 
During  Session 

•  Manipulate  data  and  share  product  information 
with  participant  sites 

•  Make  and  save  final  product  changes 

•  Record  text  log  of  significant  audio  discussion, 
decisions  with  rationale,  actions,  suspenses,  and 
product  hold  or  approval  status 

After  Session 

•  Save  log  and  provide  to  participants 

•  Post  products  in  proper  format  for  participants, 
consumers,  and  next  phase  of  process 

Product  Developer :  The  product  developer  provides 


20  A  site  may  have  multiple  roles  or  role  may  be  shared  by  multiple 
sites. 
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product  information  to  the  information  coordinator 
before  or  during  sessions. 

Before  Session 

•  Perform  detailed  development,  research,  and 
analysis 

•  Address  data  and  information  shortfalls 

•  Develop  product  for  session  review 
During  Session 

•  Obtain  priorities  from  lead 

•  Provide  rationale  or  explain  product  information 
After  Session 

•  Work  action  items  assigned  from  lead  site 

Product  and  Information  Management:  The 
information  coordinator  works  with  the  process  owner 
or  delegate  and  the  general  systems  engineers  on  the 
following  tasks. 

•  Select  mission  application  software  to  use  and 
develop  standard  product  templates  to  organize 
information  that  is  the  focus  of  the  collaborative 
session 

•  Determine  information  used  in  session, 

information  structure,  and  information  repository 
location  for  products 

•  Determine  product  format 

•  Develop  an  information  flow  and  structure  to 
transfer  and  hold  product  information  between 
various  stages  in  the  process 

•  Develop  information  change  procedures  to  ensure 
the  currency,  accuracy,  and  integrity  of  the 
information.  Information  change  procedures  for 
shared  information  is  critical  to  provide  version 
control  and  conduct  successful  collaborative 
sessions. 

The  importance  of  information  management  to  conduct 
productive  collaborative  sessions  cannot  be  over 
emphasized. 

Nurturing:  Nurturing  is  an  important  human  factor 
component  to  successfully  apply  collaboration. 
Nurturers  assist  in  institutionalizing  collaboration  as  an 
alternative  method  for  supporting  mission  processes 
and  inserting  additional  collaborative  techniques  and 
capabilities  as  military  operators’  work  schedules 
permit.  As  a  minimum,  a  nurturing  team  should  have  a 
general  systems  engineer  and  military  operator  to 
bridge  across  each  other’s  disciplines.  The  resources 
and  time  spent  with  operators  on  modifying  mission 
processes  and  applying  technology  should  be  equal  to 
or  greater  than  resources  and  time  spent  on  technology 
development  and  fielding  issues  to  improve  overall 
mission  benefit.  “Buttonology”  training  is  not 
sufficient.  Mission  process  training  proved  to  be 
beneficial  in  executing  collaborative  sessions  with 
mission  groups. 

Personnel  Data,  and  Preparation:  Successful 
collaborative  sessions  are  not  guaranteed.  Conducting 
successful  collaborative  sessions  is  similar  to  holding 


productive  meetings.  The  existence  of  current, 
accurate,  and  validated  information  is  important.  The 
right  functional  combination  of  trained  personnel  to 
prepare  for,  participate  in,  and  contribute  to 
collaborative  sessions  are  paramount  to  conducting 
successful  sessions,  making  good  decisions,  and 
ensuring  product  quality. 

Session  Participation  Control:  Mission  process  owners 
control  collaborative  session  participation.  Process 
owners  or  designates  must  be  able  to  immediately  add 
or  delete  participants  to  sessions.  Participants  required 
network  access  accounts  on  a  properly  configured 
workstation.  This  is  important  since  operators  may 
deploy  to  various  sites  and  numerous  augmentees  assist 
locally  or  in  a  federated  manner  during  crises.  Battle 
rhythm  and  mission  requirements  often  dictate 
expedient  addition  or  deletion  of  participants. 
Participants  require  group  or  organizational  accounts  to 
improve  collaboration  benefits  to  some  mission 
processes. 

Some  reviewers  indicated  that  using  collaboration  in  a 
NATO  environment  may  be  less  open  than  U.S.  only 
sessions.  Controls  on  participation,  including  the 
country  level,  need  to  be  provided  with  the  appropriate 
privacy  and  security  mechanisms  to  allow  flexible 
selectivity  to  address  releaseability  issues  during 
collaborative  activities. 

Accreditation  and  Approval  to  Operate  Lessons 
Learned  and  Best  Practices 

Obtaining  accreditation  and  “approval  to  operate”  for 
collaboration  requires  dedicated  effort.  Personnel  who 
provide  accreditation  and  give  “approval  to  operate” 
for  information  systems  must  be  included  in  the  team 
from  the  beginning.  Sharing  information  and 
collaborating  over  networks  requires  close  cooperation 
with  information  security  representatives  to  implement 
procedural  and  technical  solutions  that  satisfy  both 
collaboration  and  accreditation  requirements. 


Summary 

The  United  States  European  Command  (USEUCOM) 
is  applying  collaboration  successfully  in  an  operational 
environment  to  support  deliberate  and  crisis  planning, 
and  operations.  According  to  Operation  Allied  Force 
operators,  proper  application  of  collaboration  improves 
the  effectiveness  of  information  processes,  improves 
product  quality,  and  benefits  federated  efforts  by 
geographically  separated  partners.  Collaboration 
allows  USEUCOM’s  geographically  separated  sites  to 
work  as  a  team  and  manage  increased  battle 
management  complexity  by  mitigating  the  effects  of 
information  overload,  improving  team  decision¬ 
making,  and  synchronizing  situational  awareness. 
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USEUCOM’s  Operation  Allied  Force  lessons  learned 
clearly  show  the  requirement  to  work  with  NATO 
allies  in  a  collaborative  environment.  USEUCOM 
operators  validated  the  requirement  to  collaborate  with 
NATO  allies.  USEUCOM  senior  leadership  and  the 
targeting  community  are  leading  advocates  for  this 
requirement.  NATO  should  consider  satisfying  the 
collaboration  requirement.  An  existing  or  new 
working  group  could  be  appointed  to  energize, 
resource,  and  satisfy  this  requirement.  First,  NATO 
might  consider  satisfying  this  requirement  within  the 
targeting  community,  focusing  on  target  development 
and  nomination,  target  approval,  and  Air  Tasking 
Order  (ATO)  generation  and  management. 

USEUCOM  encountered  and  addressed  several 
challenges  and  issues  to  provide  collaborative 
capabilities  and  apply  collaboration  to  benefit  mission 
processes.  USEUCOM  Operation  Allied  Force  lessons 
learned  and  best  practices  are  provided  for  NATO’s 
consideration  and  potential  use  to  satisfy  the 
collaboration  requirement.  Technical  lessons  learned 
and  best  practices  highlight  performance,  reliability, 
and  simplicity  as  the  primary  factors  that  affect 
operator  acceptance  and  use.  Operational  lessons 
learned  and  best  practices  demonstrate  that 
collaboration  can  benefit  mission  effectiveness  when 
judiciously  applied  to  existing  or  modified  processes,  a 
process  owner  is  appointed  who  carries  out  the 
responsibilities  recommended,  and  a  Concept  of 
Operations  (CONOPs)  is  documented.  Collaboration 
does  not  replace  the  need  for  the  right  functional 
combination  of  well-trained,  prepared  personnel  who 
have  access  to  current  and  accurate  information. 

The  investments  made  and  risks  taken  by  NATO  and 
member  nations  to  improve  and  increase  collaboration 
could  provide  significant  benefit  to  NATO’s  mission 
effectiveness  and  alliance  solidarity  by: 

•  Allowing  military  resources  to  be  allocated  more 
effectively  and  used  more  efficiently 

•  Improving  alliance  coordination,  synchronization 
of  situational  awareness,  and  decision-making. 

The  “need  to  know”  constraint  must  be  balanced 
against  the  “need  to  share”  necessity  to  satisfy  time 
dependent  allied  operational  requirements  to  execute 
effective  missions. 


Attachment  1 

Primary  Collaborative  Techniques  and  Capabilities 
Used  at  USEUCOM 

Text  chat:  Participants  use  text  chat  as  one  of  the 
primary  communication  capabilities  during 
collaborative  sessions.  A  date/time-stamped,  formal 
log  is  created  and  saved  using  text  chat.  Some  users 
indicated  that  text  chat  is  used  more  than  audio  once 


experience  is  gained  using  the  system.  Each  product 
has  a  historical  log  with  key  points,  questions,  actions, 
and  current  product  status.  The  information  coordinator 
uses  text  chat  to  document  key  points  and  final 
decisions  made  using  audio.  The  historical  log  allows 
participants  to  agree  and  view  the  group’s  final 
decision.  Text  chat  allows  sites  to  provide  feedback  to 
the  information  coordinator.  For  example,  sites  notify 
the  session  information  coordinator  when  product 
information  is  in  view.  Text  chat  also  provides  an 
alternative  means  to  communicate  if  audio  is  not 
available  or  practical. 

Voice  Audio:  Participants  frequently  use  the  voice 
audio  capability  in  a  coordinated  manner  to  express 
concerns  that  were  not  easily  communicated  with  the 
text  chat  tool  or  to  discuss  contentious  and  complex 
issues.  Normal  military  radio  procedures  ensure  clarity 
and  brevity.  Audio  is  disabled  when  participants  are 
not  speaking  to  conserve  bandwidth  and  reduce 
feedback.  Audio  significantly  reduces  the  time  to 
reach  consensus  or  make  decisions  during  sessions. 
Audio  generates  interaction  and  synergy,  enhancing  the 
quality  of  products.  Overall,  participants  indicate  text 
chat-only  sessions  slow  and  reduce  the  information 
exchange  among  key  participants.  However, 
productive  sessions  have  been  conducted  using  text 
chat  only.  Audio  better  enables  the  leader  to  direct 
sessions.  The  leader  uses  audio  to  guide  the  session 
while  participants  provide  responses  using  text  chat. 
Participants  use  audio  to  direct  requests  to  the 
information  coordinator  for  information  or  product 
manipulation.  Operators  emphasize  that  the 
combination  of  text  chat  and  audio  used  in  the  right 
proportion  is  extremely  beneficial  and  that  both 
capabilities  are  required. 

Application  Sharing  Capability  to  Share  Products  and 
Information:  Application  sharing  provides  products  and 
information  to  participants  during  collaborative 
sessions.  Single-point  controlled  application  sharing  is 
when  a  single  collaborative  session  participant,  the 
information  coordinator,  shares  a  digitized  product  or 
relevant  information  with  participants.  The  information 
coordinator  manipulates  the  application  and  modifies 
the  product  after  the  proposed  changes  are  discussed 
via  text  chat  and/or  audio,  and  approved  by  the  leader. 
The  information  coordinator  does  not  give  up  control 
of  the  application.  Reasons  for  using  single-point 
controlled  application  sharing  are  to: 

•  Conserve  bandwidth  and  reduce  latency  effects 

•  Allow  control  of  session  and  reduce  chaos 

•  Allow  participation  of  new  operators  on  different 
shifts  from  multiple  sites  with  varying  computer 
skills 

•  Keep  techniques  simple 

•  Reduce  training  requirements 


Operation  Allied  Force  military  operators  who 
participated  in  collaborative  sessions  commented  on 
some  of  the  benefits  of  application  sharing. 
Application  sharing: 

•  Maintains  standard  or  native  product  file  format 
and  eliminates  the  need  to  perform  format 
translation  (e.g.,  From  PowerPoint  format  to 
whiteboard  format  and  back  to  PowerPoint  format 
after  collaborative  session).  Inflexible  operational 
process  timelines  do  not  allow  personnel  to  pre 
and  post-process  information.  Operators  prefer  to 
work  in  the  process  product’s  native  format,  make 
changes,  and  save  to  the  designated  information 
repository. 

•  Allows  use  of  core  mission  application  software 
and  provides  the  ability  to  take  advantage  of  all  the 
capabilities  (e.g.,  zoom)  resident  in  the  application 
being  shared 

•  Allows  use  of  the  primary  information  repository 

•  Allows  participants  to  see  the  information 
coordinator’s  cursor  movement.  This  is  beneficial 
in  providing  feedback.  Participants  know  that  the 
information  coordinator  is  manipulating  the 
application.  Using  the  whiteboard  does  not 
provide  this  feedback. 

Whiteboard 

Some  sessions  import  imagery  into  the  whiteboard  to 
facilitate  discussions.  Participants  share  and  annotate 
images  using  whiteboard  capabilities  to  coordinate 
tasking  and  synchronize  understanding.  More  team 
discipline  and  training  is  required  when  using 
whiteboard  capabilities  compared  with  single-point 
application  sharing  since  any  participant  may  modify 
the  whiteboard  at  any  time.  The  changes  made  during 
the  session  have  to  be  repeated  in  the  native  application 
if  the  whiteboard  does  not  support  saving  in  the 
product’s  native  application  format. 

Web-Based  Technology 

Web-based  access  to  product  information  is  a  key 
component  of  USEUCOM’s  application  of 
collaboration  to  mission  processes.  Providing 
information  on  web-sites  significantly  reduces  staff 
workload  by  decreasing  the  number  of  duplicate  email 
or  phone  requests  for  information.  Rapid  access  to  the 
most  current  and  accurate  information  for  intelligence 
and  operations  planning  and  execution  provides  a 
significant  benefit  according  to  Operation  Allied  Force 
participants.  Web-based  forms  and  databases  reduce 
workload,  limit  ambiguities,  and  decrease  the  number 
of  information  updates. 
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